Extending X3D for Augmented Reality Fifth AR Standards Group Meeting Anita Havele Executive Director, Web3D Consortium www.web3d.org [email protected] March 19, 2012

• • • • •

March 19, 2012

Overview AR WG Update ISO SC24 AR Update OGC 3DPIE Report Upcoming events

Extending X3D for AR

Open Standards www.web3d.org

• • • •

Portability Durability IP-independence ISO Approved

March 19, 2012

Extending X3D for AR

About the Web3D Consortium

We are an International non-profit, member-funded, industry group Our members span business, academia, government and the military We maintain a royalty-free, ISO standard “X3D” for web based 3D. March 19, 2012

Extending X3D for AR

X3D - ISO Standard

X3D an event and runtime model build for the Web March 19, 2012

Extending X3D for AR

: the Standard Scene graph Scene graph for real-time interactive delivery of virtual environments over the web: • • • • •

Meshes, lights, materials, textures, shaders Integrated video, audio Animation Interaction Scripts & Behaviors

Event ROUTE Sensor

• Multiple encodings (ISO = XML, VRML-Classic, Binary) • Multiple Application Programming Interfaces (ISO = ECMA, Java) • X3D Version 3.3 in draft mode includes Volume rendering, CAD and Geospatial components. March 19, 2012

Extending X3D for AR

Web3D Collaboration & Convergence XML HTML5 AR POI

Web3DS CityGML KML

Mpeg 4 WebGL Collada OPenGL

N-D Presentation State Volume data

• VRML and X3D are important parts of other specifications • We extend our standards to meet the needs of new use groups March 19, 2012

Extending X3D for AR

Adoption

March 19, 2012

Extending X3D for AR

Motivation to extend X3D for AR/MR Extend X3D capabilities to address the larger “2D + 3D” space for AR Going beyond basic geo-location based AR Virtual Objects

Real Objects

MR Continuum

VR

AR

Virtual Reality (VR) – Virtual objects/ data X3D is a well established application description language to express Virtual objects and their behaviors in 3D virtual environments Augmented Reality (AR) – Virtual objects augmented by sensors X3D has partial functionality for AR (Sensor Nodes, Viewpoint node, Camera node)

Mixed Reality (MR) – Continuum between VR and AR X3D currently lacks features needed for MR - Extend X3D accommodate “real” world objects and represent MR contents March 19, 2012

Extending X3D for AR

Our Approach to extend X3D for AR •

Extensible to existing frameworks - X3D (Scene graph) Extend Virtual Environment to implement AR - KML, OpenGIS, ARML, X3dom… Location Representation

• General/Flexible – Different AR platforms (platform independence) Mobile, Desktop and HMD – Sensors and devices Vision based, Marker based and Location based

March 19, 2012

Extending X3D for AR

Extending X3D for AR Mixed Reality

Real Environment

Augmented Reality New/Extended X3D Nodes for AR/MR

March 19, 2012

Virtual Reality

Virtual Environment

X3D existing Nodes

Extending X3D for AR

Making X3D AR/MR Capable AR Working Group started in June 2011 Goal: Extend X3D specification for AR and MR applications Co-chairs: Gun Lee (HIT Lab NZ), Yvonne Jung (Fraunhofer IGD) Tasks Requirement and Use cases – August, 2011 Comparison of X3D AR proposals - March, 2012 Development to merge proposals http://www.web3d.org/x3d/wiki/index.php/X3D_and_Augmented_Reality

March 19, 2012

Extending X3D for AR

X3D-AR Requirements Functional requirements • Supporting image (camera) sensors • Live video • Background and textures • Tracking camera motion • Camera calibration • Correct image synthesis Non-functional requirements Simplicity and extensibility of physical objects to be placed within the 3D scene. March 19, 2012

Extending X3D for AR

X3D-AR Use Cases • Augmented Reality Applications Where live video stream is shown on the background and the 3D scene is shown as registered in the physical space of the live video stream. • Augmented Virtuality (or virtual studio) Applications Where live video stream of physical objects can be placed within the 3D scene. http://www.web3d.org/x3d/wiki/index.php/X3D_AR_Requirements_and_Use_cases

March 19, 2012

Extending X3D for AR

X3D-AR Comparison of Proposals X3D AR proposals from Korea Chapter and Fraunhofer IGD

Trade off between simplicity Vs flexibility/extensibility • Korea Chapter proposes on higher level AR/MR features • Instant Reality has lower-level multi-purpose nodes Based on this comparison the AR WG is discussing how to merge the X3D AR proposals without compromising the AR requirements and use cases.

http://www.web3d.org/x3d/wiki/index.php/Comparison_of_X3D_AR_Proposals

March 19, 2012

Extending X3D for AR

ISO Activities on AR Standardization JTC 1 Approval of SC24 Scope Reorganization • Widen the scope and not limit it to computer graphics,

• Include various forms of media in the spectrum of mixed reality • Virtual Reality • Augmented Reality • Augmented Virtuality

Hence the term “Augmented Reality Continuum (ARC)” March 19, 2012

Extending X3D for AR

ISO Activities on AR Standardization SC24 Goals on ARC

WG9 Work Items • ARC Concepts and Reference Model • Terms and Definitions • Architecture Model for Generic ARC System

Independent of implementation • Proper ARC content representation ARC content representation Specific file format

March 19, 2012

Extending X3D for AR

ISO Activities on AR Standardization First WG9 Meeting was held in Seoul, Korea, Nov. 2011 Defined work items for WG9 • Standard ARC Reference Architecture • Terms and Definitions for ARC These work items (for SC24) will be submitted for approval at the next SC24 Plenary Meeting (Brussels, Aug 2012) Working Group 9 homepage: http://dxp.korea.ac.kr/SC24-WG Basic information and news

Bulletin board for on-line discussion about the work items Hope to coordinate above activities with the ARS community

March 19, 2012

Extending X3D for AR

3DPIE and X3d • X3D aligns with 3DPIE visualization goals • 3DPIE report demonstrates X3D Value • Web3D Member Contributions • Virginia Tech – 3D Blacksburg Project • Bitmanagement – BS Contact Browser • Fraunhofer – Instant Reality Browser • MBARI

March 21, 2011

Extending X3D for AR

Next Steps To Align X3D with OGC Propose new X3D work to further OGC 3D portrayal Requirements

• • • • •

Align with HTML5 approach using GeoLocation APIs Consider diverse location inputs to X3D scenes Consider X3D geospatial portability on mobile devices ARML WG collaboration in comparing AR proposals Write proposals and papers to gain external sponsor funding

March 21, 2011

Extending X3D for AR

Web3D Conference 2012 www.web3d2012.org

• Co-sponsored by Web3D Consortium, ACM SIGGRAPH and Eurographics • 4th and 5th of August 2012 in Los Angeles.coLocated with SIGGRAPH 2012 • Highlights current Capabilities and trends in interactive 3D graphics. See CFP – Deadline April 3rd •

Several Workshops ( AR Workshop) and Tutorials

• Looking forward to participation from the AR Community March 21, 2011

Extending X3D for AR

Visit us at: www.web3d.org To Join: www.web3d.org/join

Executive Director: Anita Havele Email: [email protected] Web3D Consortium Castro Street, Suite #120-490 Mountain View, CA 94041, USA Phone: +1 248 342 7662

Fax: +1 248 457 8018