BLOOP Rapid Motion Capturing using Blender and Kinect Florian Biermann Nikolaas Steenbergen Benjamin Walther-Franks TZI Universität Bremen

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Who Are We? ▸ Florian Biermann and Nikolaas Steenbergen ▸ Studied Digital Media B.Sc. at the University of Bremen ▸ Just finished their theses using Blender  “Automated Camera Placement to View Three-Dimensonal Curves“  “Timing Techniques for Three Degree-of-Freedom Input Devices“

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

What is Bloop?

''The Bloop is the name given to an ultra-low frequency and extremely powerful underwater sound detected by the U.S. National Oceanic and Atmospheric Administration (NOAA) in 1997.'' (http://en.wikipedia.org/wiki/Bloop)

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

What is Bloop?

Bloop:

Blender Loop Station

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

What is Bloop?

Bloop is a Python add-on for Blender 2.59 that brings speech controlled Motion-Capturing and Digital Puppetry to home users using Microsoft Kinect.

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Motivation ▸ Key-frame animation takes a lot of time ▸ Experimenting during production? No way! ▸ Other artists are unable to enjoy computer animation  Actors  Puppeteers  Simply beginners

▸ Also, in general motion capturing equipment is very expensive  Pretty much no home-user can do mo-cap  Additionally, single user mo-cap is pretty cumbersome

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Motivation

''A user should be able to fully animate a 3D character with a minimum amount of time, experience and technological knowledge required and without breaking his or her work flow by using different modes of interaction.''

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Idea

Implement a system in style of a loop station as used by musicians but with modal channels switched.

Sound

Recorded Footage

Motion

Control Input

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Idea

Implement a system in style of a loop station as used by musicians but with modal channels switched.

Sound

Recorded Footage

Motion

Control Input

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Idea

Speech commands

OSC data

Motion Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Demo

BLOOP DEMO!

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Main Features ▸ ▸ ▸ ▸ ▸

Create new mappings via gestures Calibrate these mappings Quickly record animations Layer recordings for different mappings Record animations with more than one user acting on the same character

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Commands ▸ Basic set of speech commands  Mapping  Calibration  Recording  Start  Next  Previous

Recording

Mapping

Calibration

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Technical Aspects ▸ OSC Client (Sender) with Microsoft Kinect SDK    

Microsoft Speech API Skeleton Data Bespoke OSC library (http://www.bespokesoftware.org) C#

▸ NUI Add-on  Receives OSC data  pyOSC (http://docs.ryancoyner.com/python-osc)  Python

▸ Bloop Add-on    

Manipulates the selected 3D character Reads data from NUI Add-on Modal operator based on timer Python

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

What‘s It Good For? ▸ Intended to let novice users and other artists create 3D animations  Digital Puppetry  Performance animation  Live performances

▸ Animation prototyping  Get a first impression of your scene

▸ Experimenting  Extend upon and play with it

▸ It‘s fun!

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

Download ▸ Visit our project website at:

dm.tzi.de/research/hci/bloop

▸ Available for download

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de

End

Thanks for listening! Shoot us an e-mail: [email protected] [email protected] Enjoy the other talks!

Florian Biermann, Nikolaas Steenbergen and Benjamin Walther Franks dm.tzi.de