Realtime 3D Computer Graphics Virtual Reality AVANGO
Motivation and overview
System overview Application
AVANGO: • Framework for distributed VR. • Well known programming model: • •
•
Rapid prototyping: • • • • •
• • •
Scene graph API. Data flow network for event propagation. Command line interpreter. No compilation. Runtime modifications. Debugging support. Script layer: Scheme (and others) small, simple and fast
Extensible (Scripts, C++) Transparent support for distribution Maximum performance
AVANGO Performer
Maestro
Elk Scheme
Linux / SGI IRIX
Object representation • Object attributes as ‚Fields‘ fpFile Filename Matrix Children
• •
• • •
Objects are Field-Containers. All objects support: • •
•
Generalized interface for scripting, streaming and persistence. Completely describe object state. Single- and multi-value types.
container-interface persistence-interface
Nodes are currently based on OpenGL Performer nodes.
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Event model and object class hierarchy
Field 1
Field 1
Field 2
Field 2
Object class hierarchy: • All nodes inherit from field container concept. • Scene graph nodes • • •
are currently based on Performer. can be linked into the scene graph and ist traversals. can be linked into the application graph and the application traversal (field change, evaluate).
• Application nodes Event model: • can be linked into the application graph and the • ‚Field connections‘ application traversal • Fields of similar type. (field change, evaluate). fpFieldContainer • connect application graph • ‚Copy-on-write‘ semantics. with objects that don‘t need • Data flow graph graphics embedding. fpNode • Evaluated once per frame. • Object notification fpGeometry fpGroup • fieldHasChanged(fpField&) • fieldHasChangedLocalSideEffect(fpField&) • evaluate() fpDCS fpSwitch fpLOD fpSequence • evaluateLocalSideEffect() fpFile Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Application Structure Sensor Field Field Field
Sensor Field Field
• Sensors
Node
Field Field
Node
Field Field Field
Node
Field Field Field
Service Service Service Service Field Connection Node Reference
• Interface objects into the real world. • Sensors are field containers. • Deliver data and events to nodes via field connections. • Are not part of the modeling hierarchy.
• Services • Encapsulate basic functionality. • Provide standard API to nodes and sensors. • Extend and enhance basic functionality without breaking node and sensor implementations.
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Example: Tracker and scripting •
•
The code below establishes the TrackerSensor- and Filenodes on the right and sets up the field route. The actual names of the nodes might differ (e.g., fpFile instead of File etc.)
DCS Matrix
TrackerSensor
Children
Matrix Station
File
Node Reference
GeoNode Matrix
...
Children
...
Field Connection GeoNode
DCS
...
Matrix
...
Children
;;; Create new Instance of Class `File` (define file-node (make-instance “File“)) (-> (-> file-node ´Filename) set-value “Banana.iv”) ;;; Add to Scene Graph (-> (-> scene-root ´Children) ´add-1value file-node) ;;; Instantiate Tracker Sensor for station ´head-6dof´ (define tracker-sensor (make-instance “TrackerSensor”)) (-> (-> tracker-sensor ´Station) ´set-value “head-6dof”) ;;; Connect Banana to Tracker, making it move. (-> (-> file-node ´Matrix) ´connect-from (-> tracker-sensor ´Matrix)) Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Distribution •
!
Process 1
Process 2
Goals • Known programming model. • Transparent support of distributed application. • Dynamic group membership • Distribution of all object attributes. • Not only transformations.
Process n
Network Database Process
Client/Server
Distributed scene graph using database replication and distribution groups: Distribution Group Process 1
Process 2
Process n
Local Database Copy
Local Database Copy
Local Database Copy
Process A a b c
Process B a b
d
Network
c
d
Replication Replicated Objects Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Local Objects
Object replication Process A
fpObject
fpObject
Field 1 Field 2
Field 1 Field 2
fpObject:Field1=Value1:Field2=Value2
Process B
fpObject:Field1=Value1:Field2=Value2
Network Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Event distributen Step 1:
Step 2:
Process A
Process B
Field 1 Children
Field 1 Children
Field 3
Field 2
Field 3
Field 2
Process A
Process B
Field 1 Children
Field 1 Children
Field 3
Field 2
Step 3: Process A
Process B
Field 1 Children
Field 1 Children
Field 3
Field 2
Field 3
Field 2
Field 3
Step 4: Process A/B Field Field11 Children Children
Field Field33
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Field Field22
Field 2
Extending the framework 1. Extending basic types: • Supply streaming operators > (). • Supply scheme binding for • representation. • access functions. 2. Extending Fields: • Take a basic type • Use SingleField and MultiField Templates • Example: field of type double:
3. Extending nodes and sensors • Derive from existing node/sensor class. • Declare fields. • Overload notification method.
Each Class is one DSO file. Multiple classes can be packed into one DSO. DSOs loaded on demand at class instantiation
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Realtime 3D Computer Graphics Virtual Reality AVANGO Development
The AVANGO shells Two AVANGO shells available: • •
• • • •
av-sh: run $AV_HOME/bin/av-sh.sh aview: run $AV_HOME/bin/aview.sh
Both shells run the same binary Shells run a scheme interpreter with AVANGO objects available Behavior and appearance is changed via command line options. Supported architectures: • •
Used to view AVANGO scenes. Used to configure output devices (CAVE, Workbench, Screen, …). Sets up the a suitable view platform for the selected output device.
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
The AVANGO shells •
Basic command line options • All parameter passed to aview.sh will be passed to av-sh.sh. • aview is just an av-sh • with some preloaded configuration options and • scheme scripts that set up the basic viewing window.
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO shell execution AVANGO task 1: • Run both shells by executing • av-sh.sh • aview.sh
“Examine the nice difference” • Run the aview with an arbitrary geometry file by just typing • aview.sh
AVANGO task 2: • aview has by default a somehow strange moving model • The –o options: All –o pairs are accessible from your scheme code • Change your aview configuration by running aview with the option –o device:mon
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
The AVANGO shells • •
• • • •
There is one basic shell binary. All configuration is done by • command line parameters. • scheme scripts. Aview can load arbitrary geometry. Remember: The shell implements a basic scheme interpreter. A scheme script can be loaded via the –f option. Different scripting languages are supported using a common scripting interface. Example Scheme script: (define myvar 1) (set! myvar 2) (define mynode (make-instance-by-name „anode“)) (-> (-> mynode ´Name) ´set-value „myname“) (fp-set-value mynode ´Name „myname“) (if (fp-option-value "verbose") (begin (printf "aview: loading setup ...\n"))) Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO - aview built in symbols • (av-help) • Display this text. • (av-reset) • Reset the viewer position. • (av-open "filename") • Load geometry from 'filename‚ and replace old. • (av-import "filename") • Load geometry from 'filename' and add to old. • (av-load "filename") • Load scheme file with overloaded scene-root. • (av-clear) • Delete scene graph. • (av-dump ) • Dump the fields of a node and their values to stdout. • (av-add ) • Add graph to root node. Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO - aview built in symbols • av-lib-vicinity • The vicinity in which the aview support files can be found. • av-data-vicinity • The vicinity in which the aview data files can be found. • av-view-sensor • The main view sensor in use. • av-main-screen • The main screen. Useful for calibration scripts. • av-viewer • The viewer node. This node is referenced by the av-view-sensor and define the viewer position. By default it is connected via field connection to the Matrix field of a fpFlyer. • av-mover • By default a fpFlyer node connected to av-viewer.
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO - aview built in symbols • av-head • If we are in a head tracked environment this is a reference to the node which represents the head. in non head tracked environments this is equal to avviewer. • av-right-eye • In a stereo projection environment these are references to the nodes whic represent the left and right eyes of the viewer. in non stereo projection environments they are equal to av-viewer. • av-stylus • If the output device supports a stylus like device this is a reference to a fpToolStylus node with a fpPointTool connected. • av-stick • If an input device in form of a joystick is supported, this is referenced here. • av-output-device • This variable can be used to make scripts output device dependent. Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO task 3: Scene graph and bindings • •
Create a basic scene graph with 4 objects and two layers. Manipulate the approp. matrix fields • Alter the position of the objects. • Play around with different values and see the results. (define node1 (make-instance-by-name „fpLoadFile“)) (define node2 (make-instance-by-name „fpGroup“)) … (fp-set-value node1 ´Filename „“) (av-add node2) (fp-add-1value node2 ´Children node1) (fp-set-value node1 ´Matrix (make-trans-mat x y z)) …
• •
There are script bindings for all necessary objects and operations. For example: • Certain required complex data types like a matrix need a scheme binding that can be manipulated from the scheme side: (mat? m) Is it a matrix? (make-mat a00 a01…a33) Create a matrix. (make-trans-mat x y z) Create a translation matrix. (mult-mat mat1…matn) Multiply the matrices 1…n. Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO task 4: simple data routing • • •
Create a scene with a fpTimeSensor and a fpText node. Connect the time-sensors time field to the text-nodes TimeString field. Load that scheme file in your preferred aview. (define mytime (make-instance-by-name „fpTimeSensor“)) (define txt (make-instance-by-name „fpText“)) (fp-set-value txt ´Name „MyTextNode“) (fp-set-value txt ´Color (make-vec4 1 0 0 1)) do some more for the txt fields Mat, Justify or DrawStyle as you wish. … (av-add …) you already should know what to to here (-> (-> txt ´TimeString) ´connect-from(-> mytime ´Time)) …
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO task 5: Creating your own objects •
•
AVANGO extension: • If you don’t find a node that suits your needs, just create a new one and make it available to the scripting layer. • You have to implement certain features in C++ to enable the AVANGO functionality. Create a new node type that gets time input and generates a scaling matrix in a certain range.
class fpNewClass : public fpDCS { FP_FC_DECLARE(); public: fpNynewClass(); //constructor virtual ~fpNewClass(); //destructor fpSFFloat TimeIn; // some fields fpSFFloat Active; // „ „ virtual void evaluate(); virtual void evaluateLocalSideEffect(); virtual void fieldHasChanged(fpField&); virtual void fieldHasChangedLocalSideEffect(fpField&); private: static pfType *classType; }; Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO task 5: Creating your own objects FP_FC_DEFINE(fpNewClass) fpType *fpNewClass::classType = 0; fpNewClass::fpNewClass(): your init values { FP_FC_START_CONSTRUCT(fpDCS); FP_FC_ADD_FIELD(TimeIn, 0.0); FP_FC_ADD_FIELD(Active, false); … FP_FC_FINISH_CONSTRUCT(); } fpNewClass::initClass(){ if (classType == NULL) { fpDCS::initClass(); classType = new pfType(fpDCS::getClassType(), „fpNewClass"); FP_FC_INIT(fpDCS, fpNewClass, TRUE, classType); } … Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
AVANGO task 5: Creating your own objects … fpNewClass::evaluateLocalSideEffect(){ double elapsed = TimeIn.getValue() - _lastTime; if ( Active.getValue()){ // do your stuff here }} fpNewClass::fieldHasChangedLocalSideEffect(field){ if (&field == &Active) // do something if (&field == &TimeIn) // do something else …
} •
Example c++ header file
•
Example c++ source file
•
Example Makefile
Realtime 3D Computer Graphics / Virtual Reality – WS 2005/2006 – Marc Erich Latoschik
Realtime 3D Computer Graphics Virtual Reality Thank You Marc Erich Latoschik AI & VR Lab Artificial Intelligence Group University of Bielefeld
