Functional Near-Infrared Spectroscopy [fNIR] …optical imaging to monitor brain activation Drexel University Optical Brain Imaging Group Drexel University, Philadelphia, U.S.A July 27, 2012 International Summer School and Workshop on Brain Dynamics, Marmaris, Turkey July 23 - 28 , 2012

Outline •

Optical Brain Imaging (fNIR) • Physiological and Physical Principles • Instrumentation and Device Designs • fNIR Signal Processing • Assessment of Cognitive Functions

• Applications • CONQUER Collaborative - Regional and Global Outreach • Human Performance, Learning and Clinical Applications

•

Overview of Field Applications • Human Performance Assessment • Depth of Anesthesia Monitoring

Physiological Principles of fNIR Neural Activity & Hemodynamic Response Neurons consume energy (glucose) when activated Oxygen is required to metabolize the glucose As clusters of neurons are activated, there is an increased need for oxygen in that area Oxygen is transported to neural tissue via oxy-hemoglobin in the blood The oxygen exchange occurs in the capillary beds As oxy-hemoglobin gives up oxygen to the neural tissue, it is transformed into deoxygenated hemoglobin

Oxy-Hb and deoxy-Hb are correlates of brain activity through oxygen consumption by neurons •David J. Heeger & David Ress. (2002)What does fMRI tell us about neuronal activity? Nature Reviews Neuroscience 3, 142-151

Physical Principles of fNIR I Photon Migration in Tissue •Source •Detector

Photons that enter the tissue undergo two types of interaction:  Scattering (cell membranes)  Absorption (Hb, HbO2, water) •H Obrig , R Wenzel, M Kohl, S Horst, P Wobst, J Steinbrink, F Thomas, A Villringer. Near-infrared spectroscopy: does it function in functional activation studies of the adult brain? International Journal of Psychophysiology 35(2-3):125-142, 2000

Physical Principles of fNIR II Optical Window in Tissue

Stimuli Presentation

Near IR

Peak Oxygenation

Initial Dip Post stimulus undershoot

− (α

+α

C HB +αHBO C HBO2 )L − (αHB L HBCHB HBO22 CHBO 2 = .G I I 0010 )

IIBB = ∆ ∆ ∆ )L ∆OD = log = ((α ∆C HB + +α ∆C αHB αHBO OD = CHBO log10 10 HB CHB HBO22 HBO22 ) L II

Modified Beer-Lambert Law

− ∆C Hb Oxygenation = ∆C HbO HbO22 Hb + ∆CHb BloodVolume = ∆CHbO HbO22 Hb

Io

I

fNIR Designs Currently Available Example CW System: Hitachi ETC-4000 MONSTIR: The UCL 32-channel time-resolved imager

IMAGENT: ISS, Inc

Drexel fNIR Instrumentation fNIR Technology Evolution Fiber-Based Full Head System Wireless system Continuous Wave Portable system

2001-08

2008-

Chance-Drexel fNIR Technology Evolution Early versions of continuous wave (CW) system

RunMan, Cogniscope NIM, Inc

Current version of CW system

fNIR System 100 fNIR Devices, LLC

Chance-Drexel fNIR Wireless Technology Evolution Early versions of wireless system (PDA based)

Wireless pediatric system (ZigBee)

Adhesive stick-on wireless system Sensors data

Programmable/ Transmitter IC Embedded in a head band with batteries

Wireless Link

fNIR Signal Processing • •

•

Raw fNIR signals are light intensity values at detectors The physiologically irrelevant data (such as respiration and heart pulsation effects) and equipment noise, etc. are first eliminated from the raw fNIR measurements Saturation, motion artifact and other noise needs to be eliminated (or excluded from analysis

Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., & Onaral, B. (2011). Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial Navigation. J Vis Exp(56), e3443. doi: 10.3791/3443

fNIR Signal Processing (cont’d) •

Motion Artifact Detection & Removal – – – – – –

Low-pass / Band-pass Filters Wavelet Analysis Independent Component Analysis (ICA) Principle Component Analysis (PCA) Coefficient of Variance related (SMAR, etc.) Optimal Filtering • • •

Adaptive Wiener Kalman

Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., & Onaral, B. (2011). Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial Navigation. J Vis Exp(56), e3443. doi: 10.3791/3443 Huppert, T. J., Diamond, S. G., Franceschini, M. A., & Boas, D. A. (2009). HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl Opt, 48(10), D280-298. Izzetoglu, M., Chitrapu, P., Bunce, S., & Onaral, B. (2010). Motion artifact cancellation in NIR spectroscopy using discrete Kalman filtering. Biomed Eng Online, 9(1), 16. Ayaz, H., Izzetoglu, M., Shewokis, P. A., & Onaral, B. (2010). Sliding-window Motion Artifact Rejection for Functional Near-Infrared Spectroscopy. Conf Proc IEEE Eng Med Biol Soc, 6567-6570. Izzetoglu, M., Devaraj, A., Bunce, S., & Onaral, B. (2005). Motion artifact cancellation in NIR spectroscopy using Wiener filtering. IEEE Trans Biomed Eng, 52(5), 934-938.

fNIR Signal Processing Artifact Cancellation (Motion Artifact Removal) Wiener Filter without Accelerometer

Adaptive Filter with Accelerometer

Subject 5, channel 16

Subject 15, Channel 12

Combined ICA & PCA Approach

2000

730nm, Correlation with Reference R=0.90 Reference Measurement 850nm, Correlation with Reference R=0.90

3500

730nm, Correlation with Reference R=0.08 Reference Measurement 850nm, Correlation with Reference R=0.34

1800

3000

1600

2500

1400 1200

2000

1000 1500

800 1000

600 500

400 0

50

100

150

200

250

300

350

400

450

0

50

100

150

200

250

300

350

400

Izzetoglu M, Devaraj A, Bunce S, Onaral B, “Motion Artifact Cancellation in NIR Spectroscopy Using Wiener Filtering”, IEEE Transaction on BME

450

Assessment of Cognitive Function Target Categorization

fMRI Results Infrequent Events Transiently Activate Human Prefrontal and Parietal Cortex as Measured by Functional MRI Gregory Mccarthy, Marie Luby, John Gore, And Patricia Goldman-Rakic. J. Neurophysiol. 77: 1630–1634, 1997.

Courtesy of Dr. Scott Bunce

Target

Assessment of Cognitive Function Target Categorization (Cont’d) fNIR Results

fNIR Data Right

Left

Izzetoglu M, Izzetoglu K, Bunce S, Ayaz, H, Devaraj A, Onaral B, Pourrezaei K “Functional Near Infrared Neuroimaging”, IEEE Transaction on Neural Systems and Rehabilitation Engineering – Special Issue on Neural Engineering

Assessment of Cognitive Function Working Memory (n-back Task)

Assessment of Cognitive Function Working Memory (n-back Task) Averaged Mean Oxygenation for 9 Subjects 0.150 0.100 0.050 0-back 1-back 2-back 3-back

0.000 1 -0.050 -0.100 -0.150 -0.200

.2

Mean Oxygenation Change

.1

0.0

-.1 0

NBACK

1

2

3

n-back results performance>90%

Results agree with fMRI results from Smith & Jonides, 1997. Izzetoglu M, Bunce S, Onaral B, “Single Trial Hemodynamic Response Estimation in Event Related fNIR Spectroscopy”, Proc. of Optical Society of America Conference, 2003

CONQUER CollabOrative Cognitive Neuroengineering & Quantitative Experimental Research Collaborative

A unique brain function research facility that houses human experiment rooms equipped with  brain activity monitoring modalities, such as near-infrared based functional optical brain imaging [fNIR] electroencephalogram [EEG]  experimental protocol design tools and simulation technologies

CONQUER CollabOrative (Cont’d) Cognitive Neuroengineering & Quantitative Experimental Research Collaborative

Strong emphasis on collaborative research in the field of Human Performance and Training Cognitive workload assessment… Brain integrated Learning (Neuro-Learning) Math, Reading, Problem solving Motor Learning Learning Disabilities Brain Computer Interface (BCI) Neurorehabilitation in Traumatic Brain injury Pediatric monitoring, etc…

Global Outreach in Functional Optical Brain Imaging

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

School of Biomedical Engineering, Science & Health Systems, Drexel University. Philadelphia, PA, USA. College of Nursing and Health Professions, Drexel University. Philadelphia, PA, USA. Middle East Technical University. Ankara, Turkey. Department of Psychiatry, M.S. Hershey Medical Center, Penn State College of Medicine, USA. Hershey, PA, USA. Temple University. Philadelphia, PA, USA. The Edmond J. Safra Brain Research Center for the Study of Learning Disabilities, University of Haifa. Haifa, Israel. Department of Psychology, Drexel University. Philadelphia, PA, USA. University of Maryland. College Park, MD, USA. LeBow College of Bussiness, Drexel University. Philadelphia, PA, USA. Antoinette Westphal College of Media Arts & Design, Drexel University. Philadelphia, PA, USA. College of Arts and Sciences, Drexel University. Philadelphia, PA, USA. College of Medicine, Drexel University. Philadelphia, PA, USA. Yeshiva University. New York, NY, USA. The Hebrew University of Jerusalem. Jerusalem, Israel. Infrascan Inc. Philadelphia, PA, USA. Rowan University. Philadelphia, PA, USA. Riddle Memorial Hospital. Media, PA, USA. Center for Autism Research, Children Hospital of Philadelphia (CHOP). Philadelphia, PA, USA. School of Public Health, Drexel University. Philadelphia, PA, USA. Office of Regulatory Research Compliance, Drexel University. Philadelphia, PA, USA. Electrical and Computer Engineering, Drexel University. Philadelphia, PA, USA. Office of Technology Commercialization, Drexel University. Philadelphia, PA, USA. Advanced Technology Laboratories (ATL), Lockheed Martin Corporation. Bethesda, MD, USA. fNIR Devices, LLC. Philadelphia, PA, USA. BIOPAC Inc. Goleta, CA, USA. Pulsar, Inc. Philadelphia, PA, USA. Baxter, Inc. Deerfield, IL, USA.

28 Environmental Tectonics Corporation (ETC), Southampton, PA, USA. 29 Telemedicine and Advanced Technology Research Center (TATRC). Fort Detrick, MD, USA. 30 Atlantic City International Airport, Federal Aviation Administration (FAA) William J. Hughes Technical Center. Atlantic City, NJ, USA. 31 George Mason University. Fairfax, VA, USA. 32 University of North Carolina Chapel Hill, NC, USA. 33 University of Pittsburgh, Pittsburgh, PA, USA. 34 NASTAR: The National Aerospace Training and Research Southampton, PA, USA. 35 Embry-Riddle Aeronautical University. Daytona Beach, FL, USA. 36 National Board of Medical Examiners (NBME) Philadelphia, PA, USA. 37 University of Seville. Seville, Spain. 38 Brain Injury Rehabilitation Centre. Seville, Spain. 39 National Hospital of Paraplegics. Toledo, Spain. 40 Polytechnic University of Milan. Milan, Italy. 41 Bukkyo University. Kyoto, Japan. 42 Modeling and Simulation Research and Development Center (MODSIMMER), METU. Ankara, Turkey. 43 Dokuz Eylül University. Izmir, Turkey. 44 Ankara University - Beyin Araştırmaları Uygulama ve Araştırma Merkezi. Ankara, Turkey. 45 Walter Reed Army Medical Center (WRAMC). Washington DC, USA. 46 University of Gothenburg. Gothenburg, Sweden. 47 School of Life Sciences and Biotechnology, Shanghai, Jia Tong, University. Shanghai, China. 48 Shanghai Institute of Mental Health, Shanghai JiaoTong University. Shanghai, China. 49 Hangzhou Dianzi University. Hangzhou, China. 50 Huazhong University of Science and Technology. Wuhan, China.

Optical Brain Imaging Human Performance Assessment…

Human Performance Assessment Portable brain imaging technology can “bring more of brain on task”*

Air Traffic Controller

Cognitive Workload Monitor Brain Activity

Crew can manage & maintain high levels of cognitive information processing

Safe & Effective

UAV Ground Operator

Piloting

Training and Workload Monitor

Monitor

Safe & effective performance *DARPA: Dylan Schmorrow, Ph.D. LCDR, MSC, US Navy, Program Manager, ITO

Pilot

Expertise Development Monitor

Air Traffic Controller FAA Next Generation Air Transportation System (NextGen) Study Similar to SESAR (Single European Sky ATM Research) Program

ATC part-task Communication type: 

Voice Comm and Data Comm

Workload manipulated by number of aircrafts in the sector: n= 6,12,18 16-channel fNIR sensor

UAV Safe Piloting US Army Medical Research and Materiel Command (USAMRMC) The Telemedicine & Advanced Technology Research Center (TATRC)  UAV ground controller’s expertise level, cognitive workload and situational awareness are critical.  Operation failures due to human error (60% - 80% )  ground flight crew’s mental workload increase  inadequate training

UAV Safe Piloting

Integration of the functional brain monitoring system with  training procedures  ground control systems

objective measure of expertise level to validate adequate training  real-time monitor of mental workload 

safe operation of UAVs…

Optical Brain Imaging Brain integrated Learning… “Brain in the Loop”

Language Learning & Learning Disability  Brain activity during language learning (Hebrew,…)  Brain activity of regular and dyslexic readers  Learning enhancement …identifying processing similarities and differences between special populations and control groups Director: Prof. Zvia Breznitz

…Lexical Decision Task

Simulation & Multimedia Application Brain Activity Monitoring in VR Environment

Optical Brain Imaging Clinical Solutions…

Optical Brain Imaging… Near Infrared Based Platform

Traumatic Brain Injury >> Portable Near-Infrared Technology for Detection of Hematoma

Drexel Optical Brain Imaging Team

Hasan Ayaz, PhD BIOMED

Meltem İzzetoğlu, Ph.D BIOMED

Baruch Ben Dor, PhD BIOMED

Kurtulus Izzetoglu, PhD BIOMED

Banu Onaral, PhD BIOMED

InfrascannerTM

…Solutions

Life Saving… Near Infrared Based Platform

Neurorehabilitation >> Assessment of Cognitive Impairment and Recovery Following Traumatic Brain injury (TBI)

Anna C. Merzagora, PhD BIOMED

Maria Schultheis, PhD Psychology, CoAS

Healthy Subject

TBI Patient

Banu Onaral, PhD BIOMED

…Solutions

Life Saving… Near Infrared Based Platform

Depth of Anesthesia >> Portable fNIR system for monitoring depth of anesthesia to prevent awareness during surgery Patient wearing fNIR Sensor

Kurtulus Izzetoglu, Ph.D BIOMED

George Mychaskiw, MD Anesthesiology, COM

fNIR Awareness Monitor

James Meltem Reynolds, MD, COM İzzetoğlu, Ph.D BIOMED

Kambiz Pourrezaei, PhD BIOMED

Banu Onaral, PhD BIOMED

…Solutions

Optical Brain Imaging… Near Infrared Based Platform

Cognitive Aging fNIR Roee Holtzer Albert Einstein CoM

Meltem Izzetoglu, PhD BIOMED

Brain activity monitor while walking

Kurtulus Izzetoglu BIOMED Joe Verghese, MBBS, MS Albert Einstein, CoM

Subject wearing fNIR during attention task

>> Assessment of cognitive functions in natural settings Banu Onaral, PhD BIOMED

…Solutions

Optical Brain Imaging… Near Infrared Based Platform

Autism, Schizophrenia… Functional Optical Brain Imaging Sensor

Drexel Optical Brain Imaging Team

Scott Bunce, PhD Psychiatry, Penn State University COM

Banu Onaral, PhD BIOMED

Kambiz Pourrezaei, PhD BIOMED

Personality and psychiatric disorders...

…Solutions

Optical Brain Imaging… Near Infrared Based Platform

ALS, Spinal Cord Injury, Paralysis Target Patient Population: • • • •

Amyotrophic Lateral Sclerosis (ALS) Spinal Cord Injuries/Paralysis Cerebral Palsy, Muscular Dystrophy Brainstem Stroke, and others…

Terry HeimanPatterson, MD Neurology, COM

Patricia Shewokis, PhD Nursing, CoNHP

Hasan Ayaz, PhD BIOMED Banu Onaral, PhD BIOMED

>> Brain Computer Interface (BCI)

…Solutions

Life Saving… Near Infrared Based Platform

Depression, Chronic Pain, Stroke… >> To investigate the neurophysiological effects of manipulation of cortical excitability by

Guglielmo Foffani, FUHNPAIIN

Transcranial direct current stimulation (tDCS) device

Antonio Oliviero FUHNPAIIN

Anna C. Merzagora, PhD Student BIOMED

Kambiz Pourrezaei, PhD BIOMED

Banu Onaral, PhD BIOMED

…Solutions

Life Saving… Near Infrared Based Platform

Pediatric Monitoring >> Functional Optical Brain Imaging Sensor

Arye Rosen, PhD BIOMED

Harel Rosen, MD Neonatology, CHOP

fNIR Sensor for Neonates

Meltem Izzetoglu, PhD BIOMED

Kambiz Pourrezaei, PhD BIOMED

Banu Onaral, PhD BIOMED

…Solutions

Funding Sources

Optical Brain Imaging Overview of two application areas

Human Performance Assessment Objective To use physiological measures based on fNIR to predict changes in cognitive workload during a complex cognitive task. Hypothesis Blood oxygenation measured from the dorsolateral prefrontal cortex via fNIR rises with increasing task difficulty and sustained cognitive effort, is positively correlated with performance measures.

Human Performance Assessment Specific Aims

 Establish the relationship between cognitive workload, the participant’s performance, and changes in blood oxygenation levels within the dorsolateral prefrontal cortex  Determine the effect of divided attention as manipulated by the secondary component of the task.

Human Performance Assessment Method  8 healthy participants  Studies organized by the Defense Advanced Research Projects Agency (DARPA) - Augmented Cognition Program

Human Performance Assessment Task

Warship Commander Task (WCT)

The WCT has been developed by Pacific Science & Engineering Group under the direction of Space and Naval Warfare Systems Center – San Diego – as a cognitive multitasking environment to simulate the military commands and tasks

Human Performance Assessment Results – fNIR vs task load 12 10 8 6

OXY

4 2 0 -2

6

12

18

-4 -6

24

In support of our primary hypothesis a main effect for wave size across both hemispheres (F = 16.24, p