Cognitive Modeling: Multitasking & Workload

Wrapup, Questions & Exam Info Felix Putze, Dominic Heger 14.7.2011 Lecture „Cognitive Modeling“ SS 2011

1/26

What did we cover? • Introduction

Cognitive Modeling: Multitasking & Workload

• Cognition, Behaviorism, Cognitive Science, cognitive revolution, Turing test, neurological basics, measuring brain activity

• Cognitive architectures • Definition, requirements, types of architectures, ACT-R (chunks, production rules, goal, utility, …), evaluation of architectures, PSI (motivation, emotion, …)

• Memory Modeling & Knowledge Representation • Types of memory, three different memory models, knowledge representation (logic, frames, semantic networks, neural networks, Bayesian networks)

• Empirical Cognitive Models • General architecture (multimodal sensor data, preprocessing, feature extraction, recognition, fusion), integration with cognitive architectures 2/26

What did we cover? • Multitasking & Workload

Cognitive Modeling: Multitasking & Workload

• Dual tasks, Wickens’ multi-resource model, Threaded cognition, measuring workload, predicting workload

• Attention & Visual Perception • Bottom up vs. top down processing, Broadband’s filter theory, Anne Triesman’s feature integration, iCub’s attention model, guided search model

• Game Theory & Reinforcement Learning • Utility functions, game theory, Nash equilibrium, Markov decision process, Q-Learning, Dopamine

• Affective Models • Affective computing, Emotion models (descriptive models, appraisal theories), Big Five personality theory, affect recognition, affect expression

3/26

What did we cover?

Cognitive Modeling: Multitasking & Workload

• Human Learning & Machine Learning • Differences between HL and ML, biological vs. artificial neural nets, Hebbian learning, hopfield nets, conditioning, programming by demonstration, human learning theory, Leo‘s social interactive learninig, learning in ACT-R

• Human Behavior Models • Huitt‘s systems approach to human behavior, Emotion and decision making (four theories), Behavior in social context, Social network analysis

4/26

Exam

Cognitive Modeling: Multitasking & Workload

• Oral exam, ~15 minutes • If you haven‘t done already, make an appointment with Frau Scherer ([email protected]) • New appointments will not be available before the start of next semester

• How to prepare? • Read the slides again, together with your notes from the lecture • Additional material may help to get a deeper understanding • Form learning groups, simulate exam situations together

5/26

Cognitive Modeling: Multitasking & Workload

What do I need to learn? • The exam will be based on the content of the slides as well as the lecture (of course you did take notes!) • However, it is not sufficient to just memorize every word on the slides • You should be able to explain any concept mentioned on the slides. If you lack the necessary background, use additional resources to cover this ground • You will get asked transfer questions which ask for things which are not explicitly on the slides (e.g. think of example for a concept, compare two methods, argue how two approaches can be combined, …)

• Yes, we may ask formalisms (e.g. formulas, algorithms, …) • There are not that many, after all

6/26

How to make a good impression • For important concepts, think of examples and applications that were not given in the lecture Cognitive Modeling: Multitasking & Workload

• Shows that you have really understood the material

• This is not a history test, but: Know the important names and dates (e.g. when did the cognitive revolution start?) • Find connections between the lectures on different topics, try to have a big picture of all the presented building blocks • Think of applications for developers of computer systems or designers of man-machine interfaces • Know the technical terms (e.g. names of experiments, methods, models) • Helps you to give precise, on-the-spot answers • Remember that you only have 15 minutes to show what you have learned! 7/26

Demo Questions

Cognitive Modeling: Multitasking & Workload

• On the next slides, we give some questions on important topics of the lecture • They are not (!!!) representative for the true exam questions • They do not cover everything that will get asked • The final questions may be easier, harder or in a completely different style

• However, if you are well-prepared, you should be able to answer them without consulting your material • Some questions may take more than a moment to come up with a good answer

8/26

Cognitive Modeling: Multitasking & Workload

• History: Describe the relation of Behaviorism and Cognitive Science during the cognitive revolution and today! • Game Theory: Consider the game “Chicken”: • http://en.wikipedia.org/wiki/Chicken_(game)#Popular_versions (first, only read the introduction) • How does the utility function of both players look like? • Is there a dominant strategy? How many deterministic Nash equilibria are there? (Bonus: How does optimal play look like?)

• Reinforcement Learning: Discuss which of the general rules on human learning also hold for Reinforcement Learning as described on the slides! • Multitasking: Analyze a dual task using Wickens’ multiresource model! • a) Driving vs. typing in a text message on a mobile phone • b) Chatting on the phone vs. solving Rubik’s cube 9/26

Cognitive Modeling: Multitasking & Workload

• General: A customer asks you to improve his virtual learning environment (a 3D avatar teaches math by asking questions, evaluating answers and gives hints) • How could you use cognitive models for this task? • Which models would you propose as most promising?

• Cognitive architectures: Describe how the learning of production rules in ACT-R models automation of behavior! • Appraisal models: Consider different variants of a exam situation: • a) Taking the exam unprepared, confronted with unexpected questions • b) Taking an exam well prepared with a supportive examiner • Model the experienced emotions using both the OCC model and Scherer’s model. Is one model better suited for the task then the other? In what way?

10/26

Cognitive Modeling: Multitasking & Workload

• Human/Machine Learning: You have a Hopfield net with 3 neurons and you train it with the examples (1 1 1), and (-1 -1 -1). Describe what recall happens in the 3 conditions when you activate one neuron and deactivate the others! • Attention/Learning: Describe how attention modeling can help a machine in learning? • Human/Machine Learning: What changes occur in Leo’s Task model when you teach him “turn one button on, off, and on again”? • Attention: Compare what you’ve learned about the human visual system to David Marr’s perception model! • Attention: Explain how Wolfe’s Guided Visual Search can be implemented in a humanoid robot. Describe a situation where the robot’s behavior would be different than the one of an iCub. When can top-down knowledge improve performance? 11/26

Cognitive Modeling: Multitasking & Workload

Student Research at CSL: Workload Prediction • Want to estimate mental workload of the user • Model the human mind as queuing network • Calculate throughput and determine bottlenecks • Available as study-, diploma-, bachelor- or master thesis • Necessary: Good programming skills (pref. Java) and interest in researching cognitive models • Contact: [email protected] 12/26

Student Research at CSL: Gaze+EEG

Cognitive Modeling: Multitasking & Workload

• From recorded signals of a person, we can draw conclusion on its visual attention: • From gaze, we can estimate the spatial focus of attention • From gaze patterns, we can estimate the current visual task (e.g. tracking, searching, …) • From EEG, we can estimate attention level and sudden stimuli

• Combine those signals to create a “neuro-visual mouse” • From a video stream, a person has to identify and mark certain items • Mouse interaction has proven inefficient, gaze + gutton pressing better • Your task: Combine EEG and natural gaze to identify the objects which draw the person’s attention  no manual interaction required anymore!

• Available as diploma or master thesis • In cooperation with the Fraunhofer IOSB • Contact: [email protected] 13/26

+

Student Research at CSL: CM+EEG

Cognitive Modeling: Multitasking & Workload

• For HCI, it is relevant to detect whether a decision situation for the user is routine or novel/difficult • We can estimate this by comparing EEG patterns of trained and untrained situations • We can estimate this using a cognitive model based on Reinforcement Learning (entropy of score distribution, frequency of state visits, …)

• The goal of this work is to bring together both approaches: • • • •

In a cognitive multi-step decision task (maze solving) Model the task using a RL approach Record EEG of people actually solving this task Combine the prediction of the model and the estimation from the EEG

• Available as study-, diploma-, bachelor- or master thesis • In cooperation with the institute of psychology in Heidelberg • Contact: [email protected] 14/26

Student Research at CSL: Feature Selection

Cognitive Modeling: Multitasking & Workload

• Existing biosignal processing framework provides a large set of different features (esp. for EEG) • Each feature captures a specific property of the signal • Most features calculated for single electrode  multiply number by 16

• For each application, we need to identify a reliable and robust feature set for classification • Detection of motor activity requires other features and locations than estimation of cognitive workload

• Goal of this work: Improve feature selection • A number of established methods are available • Task is: systematic evaluation, implementation of additional methods and their extension to increase stability  e.g. penalize features which only work well for few participants/situations

• Available as study-, diploma-, bachelor- or master thesis • Contact: [email protected] 15/26

Cognitive Modeling: Multitasking & Workload

EEG based Emotion Recognition • Entwicklung eines multimodalen (EEG, PPG, EDA) Erkennungssystems für Emotionen • Emotionsmodell von Russell (V, A) • Basierend auf Erkennerframework (Matlab) • Ziel: Extraktion von geeigneten Merkmalen aus dem EEG zur zuverlässigen Klassifikation von Emotionen

• [email protected] 16/26

Cognitive Modeling: Multitasking & Workload

Robot Programming by Brain Activity • Normalerweise werden BCIs für einfache vorgestellte Bewegungen verwendet (z.B. Cursorsteuerung) • Ziel: Analyse von komplexen Bewegungen mittels Brain Computer Inferfacing Techniken

• Können komplexe, real ausgeführte Bewegungen erkannt und modelliert werden? • Wie kann man solche BCI Informationen für das Programmieren durch Vormachen nutzen? • SFB588 – Humanoide Roboter • [email protected]

17/26

Cognitive Modeling: Multitasking & Workload

ECoG based Speech Recognition • Spracherkennung aus invasiv gemessener Gehirnaktivität • Datenanalyse von ECoG Daten • Bleeding edge research topic

• Kenntnisse in Mustererkennung, Datenanalyse, und Matlab sind von Vorteil • Projekt mit a*star Singapur und Wadsworth Center Albany • [email protected] 18/26

Cognitive Modeling: Multitasking & Workload

Diplom- oder Masterarbeit Aktuell: Diplom- oder Masterarbeit in der EMG-basierten Spracherkennung! Ihr wisst alles über Akustik, GMMs, HMMs, …?  Wie wär‘s mal mit etwas Neuem? Am CSL erforschen wir seit Jahren sehr erfolgreich die Erkennung von Sprache mit elektromyographischen Signalen, die Muskelaktivität abbilden. • • • • •

Unser aktuelles Ziel: Bessere Erkennung von paralleler Aktivität Gut geeignet für DA oder MA Vorkenntnisse: Grundlagen der Programmierung, Verständnis für Algorithmik Selbständiges Arbeiten im Team mit guter Betreuung, eigene Aufnahmen erwünscht Bitte melden bei: Michael Wand ([email protected])

19/26

Studien/Diplom/Bachelor/Masterarbeiten

Cognitive Modeling: Multitasking & Workload

Ein weiteres heißes Thema in der EMG-basierten Sprachverarbeitung! • Experimente mit Elektrodenarrays • Elektrodenarrays/Multielektroden geben eine sehr schöne Repräsentation des Muskelsignals! • Viele interessante Experimente möglich! • Spracherkennung + Sprachsynthese, neue Themen ab ca. September • Bei Interesse bitte melden bei [email protected] oder [email protected]

20/26