Java™ Stream Assembly API Programmer’s Guide Java Stream Assembly API, Version 1.0, Community Draft Java 2 Platform, Standard Edition, Version 1.4

Sun Microsystems, Inc. 901 San Antonio Road Palo Alto, CA 94303 U.S.A. 650-960-1300 December 2004

Copyright © 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, CA 95054, U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at http:// www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and in other countries. This document and the product to which it pertains are distributed under licenses restricting their use, copying, distribution, and decompilation. No part of the product or of this document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Sun, Sun Microsystems, the Sun logo, Java, and J2SE are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. The Adobe® logo is a registered trademark of Adobe Systems, Incorporated. Federal Acquisitions: Commercial Software - Government Users Subject to Standard License Terms and Conditions.

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. Copyright © 2004 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, CA 95054, Etats-Unis. Tous droits réservés. Sun Microsystems, Inc. a les droits de propriété intellectuels relatants à la technologie incorporée dans le produit qui est décrit dans ce document. En particulier, et sans la limitation, ces droits de propriété intellectuels peuvent inclure un ou plus des brevets américains énumérés à http://www.sun.com/patents et un ou les brevets plus supplémentaires ou les applications de brevet en attente dans les Etats Unis et dans les autres pays. Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, parquelque moyen que ce soit, sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y ena. Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun. Sun, Sun Microsystems, le logo Sun, Java, J2SE, et docs.sun.com, sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays. Le logo Adobe® est une marque déposée de Adobe Systems, Incorporated. LA DOCUMENTATION EST FOURNIE "EN L’ÉTAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

Please Recycle

Contents

Preface 1.

xiii

Introduction to Assembling Media Streams Media Streams

1

Assembling Media Streams

4

Media Delivery Mechanisms Controlling Multiplexers

4

5

Feeding Data to Multiplexers 2.

Java Stream Assembly API Overview

5 7

Purpose of the Java Stream Assembly API Design Requirements Usage Scenarios

Ports

7

8

8

Fundamental Concepts Streams

1

8

9

10

Components

11

Stream Control

13

Physical Interfaces

19

Moving Data Between Components: Port Types Applications and Components Roles and Responsibilities

19

20

21

iii

Platform Provider

21

Component Provider Client Developer Packages 3.

22

24

25

Developing Applications

27

Common Development Tasks Creating a Session

27

28

Looking Up a JsaComponent Why Use a Lookup Service?

28 28

Establishing a Stream Between Two JsaComponents Controlling the Stream Flow

33

Monitoring Stream Flow Events Delivering Broadcast Video Initialization

38

39

Processing Streams

45

Changes During Operation Delivering On-Demand Video Delivery Scenarios Initialization

35

46

47

48

51

RTSP Describe Interaction RTSP Setup Interaction Playing the Video Delivering Interactive TV

51

53

59 60

Setting Up Outputs and Inputs

62

Configuring Streams, iTV Example Stream Processing, iTV Example 4.

Developing Components

68

69

Implementation Models and Strategies

iv

66

70

Java Stream Assembly API Programmer’s Guide • December 2004

29

Choosing an Implementation Strategy Modeling

71

74

Implementing Ports

77

Using Remote Method Invocation (RMI) Working with Native Objects Naming and Lookup

79

79

Providing Security Policies 5.

78

JSR-158 Web Resources

79

81

Java Stream Assembly API Specification

81

Java Stream Assembly Reference Implementation JNDI (Java Naming and Directory Interface) JNI (Java Native Interface)

81

81

82

JMX (Java Management Extensions) RMI (Remote Method Invocation)

82 82

Contents

v

vi

Java Stream Assembly API Programmer’s Guide • December 2004

Figures

4

FIGURE 1

Media Delivery Mechanisms

FIGURE 2

Relationship of Streams to Source and Sink Ports

FIGURE 3

Component Hierarchy

FIGURE 4

Stream Source and Stream Sink

FIGURE 5

Stream Control State Diagram

FIGURE 6

Java Stream Assembly API Tiers

FIGURE 7

Block Diagram: Broadcast Video Code Example

FIGURE 8

Broadcast Video Initialization Sequence

FIGURE 9

Broadcast Video Changes During Operation Sequence

FIGURE 10

Broadcast Video Stream Processing Sequence

FIGURE 11

Block Diagram: VOD Example

FIGURE 12

VOD Initialization Sequence

FIGURE 13

Sequence for Creating NetSink and Opening Ports

FIGURE 14

Creating Streams and Configuring Router Processing

FIGURE 15

Block Diagram: iTV

FIGURE 16

Initialization Sequence, iTV Example

FIGURE 17

I/O Setup Sequence, iTV Example

FIGURE 18

Stream Configuration and Processing, iTV Example

FIGURE 19

Shared Library Component Development Model

FIGURE 20

Separate Application Component Development Model

FIGURE 21

Typical UML Class Diagram

9

11 12 13 20 39

40 44

45

47 50 56 58

60 61 63 66

72 74

75

vii

FIGURE 22

viii

Typical UML Sequence Diagram

76

Java Stream Assembly API Programmer’s Guide • December 2004

Tables

TABLE 1

Comparison of Bandwidth Requirements

TABLE 2

MPEG-2 Table Controls

TABLE 3

Valid Transitions from Each State

14

TABLE 4

Descriptions of the Control Types

16

TABLE 5

Descriptions of Position Types

3

10

18

ix

x

Java Stream Assembly API Programmer’s Guide • December 2004

Code Samples

28

CODE EXAMPLE 1

Creating a Session

CODE EXAMPLE 2

Looking Up a JsaComponent

CODE EXAMPLE 3

Looking Up a JsaComponentFactory

CODE EXAMPLE 4

Creating a Stream Connection Using the Default Strategy

CODE EXAMPLE 5

Creating a Custom Strategy

CODE EXAMPLE 6

Creating a Stream Connection Using a Custom Strategy

CODE EXAMPLE 7

Controlling the Stream in the Most Basic Manner

CODE EXAMPLE 8

Using Position-Based Stream Control

CODE EXAMPLE 9

Implementing ControlContext

CODE EXAMPLE 10

Providing a BitRateOutOfRangeParams Object

CODE EXAMPLE 11

Monitoring an Overflow Event 38

CODE EXAMPLE 12

Creating a Processor Instance

CODE EXAMPLE 13

Creating and Assigning Attributes to Outputs and Inputs 41

CODE EXAMPLE 14

Getting and Checking Media Attributes 42

CODE EXAMPLE 15

Creating and Opening a File Source Instance 42

CODE EXAMPLE 16

Connecting Sources and Sinks 43

CODE EXAMPLE 17

Processing Streams, Broadcast Video Example

CODE EXAMPLE 18

Changes During Operation, Broadcast Video Example 47

CODE EXAMPLE 19

Initialization, VOD Example

CODE EXAMPLE 20

RTSP Describe Interaction, Client/Server Communication 51

CODE EXAMPLE 21

RTSP Describe Interaction, Server Code, VOD Example 52

29 29 31

31 32

33

34

36 37

41

46

51

xi

CODE EXAMPLE 22

Get MediaAttributes for Stream Content 53

CODE EXAMPLE 23

Create the RTP and RTCP Streams 53

CODE EXAMPLE 24

Creating a session object, VOD Example

CODE EXAMPLE 25

Getting the SDP Track Attributes, VOD Example 55

CODE EXAMPLE 26

Setting Up the Input Control Node, VOD Example 55

CODE EXAMPLE 27

Create and Connect Stream and Control Node, VOD Example 55

CODE EXAMPLE 28

Creating the Net Sink Factory and Instance, VOD Example 57

CODE EXAMPLE 29

Opening the RTP and RTCP Sink Ports, VOD Example

CODE EXAMPLE 30

Creating the RTP and RTCP Streams, VOD Example

CODE EXAMPLE 31

Configuring Processor Routing, VOD Example 59

CODE EXAMPLE 32

Playing the Video Selection, VOD Example 59

CODE EXAMPLE 33

Initialization, iTV Example

CODE EXAMPLE 34

Setting Up Outputs and Inputs, iTV Example

CODE EXAMPLE 35

Configuring Streams, iTV Example

CODE EXAMPLE 36

Stream Processing, iTV Example

xii

55

62 64

67 68

Java Stream Assembly API Programmer’s Guide • December 2004

57 59

Preface This programmer’s guide explains the video stream assembly environment for which the JavaTM Stream Assembly API was designed. It also describes how to use the API to perform the typical tasks that it was designed to do.

Note – This programmer’s guide draft is intended to support the community draft of JSR 158 by providing an example of the documentation that will be made available with the finished API. The code examples in this draft are based on preliminary versions of the API and should be considered descriptive rather than authoritative. To download the latest version of this document, go to http://www.jcp.org/en/jsr/detail?id=158

How This Book Is Organized Chapter 1 describes the video stream assembly environment and introduces the terms and concepts used in it. Chapter 2 introduces the Java Stream Assembly API and explains the terms and concepts that relate specifically to it. Chapter 3 provides sample code and tips for programmers developing applications that use the Java Stream Assembly API. Chapter 4 provides sample code and tips for programmers developing components that use the Java Stream Assembly API. Chapter 5 provides sample code and tips for deployers of applications and components that use the Java Stream Assembly API.

xiii

Audience Definition The Java Stream Assembly API platform defines several distinct roles in the application development and deployment life cycle: product provider, application component provider, application assembler, and deployer. These role definitions are intended as an aid in identifying the tasks performed by various parties during the development, deployment, and running of a Java Stream Assembly application. Roles can be filled by whatever personnel match an organization's actual application development and deployment workflow. Thus, each role may be performed by a different party or a single party may perform several roles. For example, a programmer may perform the roles of application component provider and application assembler. The Java Stream Assembly API roles are described below. ■

■

■

■

xiv

Product Provider—A product provider implements a Java Stream Assembly product providing features defined in the Java Stream Assembly specification (JSR 158). A Java Stream Assembly product is free to implement interfaces not specified by the Java Stream Assembly specification in an implementation-specific way. Application Component Provider—Application component providers produce the building blocks of a Java Stream Assembly application. They typically have expertise in developing reusable components as well as sufficient business domain knowledge. Application component providers need not know anything about the operational environment in which their components will be used. Application Assembler—An application assembler starts with a set of components developed by application component providers and assembles them into a complete Java Stream Assembly application. The assembler’s expertise lies in providing solutions for a specific problem domain—for example, the financial industry. Application assemblers may not be familiar with the source code of the components that they use, but they use declarative descriptors for the components in order to build applications from them. Like application component providers, they need not know anything about the operational environment in which their applications will be used. Deployer—A deployer, an expert in a specific operational environment, is responsible for deploying Java Stream Assembly components and applications into that environment. A deployer uses tools supplied by the Java Stream Assembly product provider to perform deployment tasks. A deployer installs components and applications into a Java Stream Assembly server and configures components and applications to resolve all the external dependencies declared by the application component provider and application assembler.

Java Stream Assembly API Programmer’s Guide • December 2004

The following table shows where this guide discusses each role.

Role

Where discussed

Java Stream Assembly Product Provider

See Chapters 1, 2, and 5.

Application Component Provider

See Chapters 1, 2, and 4.

Application Assembler

See Chapters 1, 2, and 3.

Deployer

See Chapters 1, 2, and 5.

Typographic Conventions Typeface

Meaning

Examples

AaBbCc123

The names of commands, files, and directories; on-screen computer output

Edit your .login file. Use ls -a to list all files. % You have mail.

AaBbCc123

What you type, when contrasted with on-screen computer output

% su Password:

AaBbCc123

Book titles, new words or terms, words to be emphasized

.

Command-line variable; replace with a real name or value

To delete a file, type rm filename.

Related Documentation Application

Title

Installation

Java Stream Assembly API Release Notes

Preface

xv

Accessing Sun Documentation Online The docs.sun.comsm web site enables you to access Sun technical documentation on the Web. You can browse the docs.sun.com archive or search for a specific book title or subject at: http://docs.sun.com

Sun Welcomes Your Comments We are interested in improving our documentation and welcome your comments and suggestions. You can e-mail your comments to us at: [email protected]

xvi

Java Stream Assembly API Programmer’s Guide • December 2004

CHAPTER

1

Introduction to Assembling Media Streams This chapter provides an introduction to media stream assembly concepts in the following sections: ■

Media Streams

■

Assembling Media Streams

This chapter introduces concepts related to media streams and multiplexing to help you understand the practical problems that the JavaTM Stream Assembly API aims to solve. This understanding will prepare you to use the API effectively.

Media Streams Stream multiplexers and media streams are at the heart of the Java Stream Assembly API. A media stream is a continuing, time-critical flow of audio, video, and data information. A multiplex is a single stream that contains multiple, interleaved media streams. Multiplexing is the process of combining several media streams in such a way that the combined signal can later be separated, or de-multiplexed, into its component parts. Multiplexing can be physical when the data packets are physically interleaved (for example, MPEG2 transport streams), or logical when the data packets are logically combined into a single session. Alternatively, the data packets can remain as separate abstract streams (multiple RTP channels in a single RTSP session). A multiplex that contains a single program is called a single program multiplex. A multiplex that contains several programs is called a multi-program multiplex. The Motion Picture Experts Group Systems (MPEG2-Systems) specification defines the concept of an elementary stream and two forms of multiplexed streams—program streams and transport streams. The elementary stream is a single flow of time-critical content such as audio, video, or data. A multiplexed stream (either a program stream or a transport stream) is a collection of elementary streams that often consists of audio,

1

video, and data. A multiplexed stream also contains tables that characterize the stream content (whether audio, video, or data) and define the address of each elementary stream. A single program transport stream (SPTS) contains the video and audio signals for one television program. A multi-program transport stream (MPTS) combines the digitized data from multiple programs in a single stream of interleaved data packets, each of which contains a fragment of the digitized data from one of the component media streams. Transport streams can be manipulated in many ways. The following are just a few examples: ■ ■

■

SPTSs may be added to or dropped from the multiplex. Information within the individual streams, such as packet IDs (PIDs), may be remapped. Input streams that do not include PIDs may have PIDs added.

Functions like the ones listed above are performed by media multiplexers. An IP (Internet Protocol) network is a packet-switched network. Hence, multiplexing in an IP network is logical. Each stream is destined for a different combination of IP address and port. The IP protocol multiplexes the packets associated with the different streams onto the physical layer (such as Ethernet). Other important concepts include the following: ■

■

An RTSP session is a complete RTSP transaction—for example, the viewing of a movie. A session typically consists of a client setting up a transport mechanism for the continuous media stream (SETUP), starting the stream with PLAY or RECORD, and closing the stream with TEARDOWN (source RFC 2326). An RTP channel is the association among a set of participants communicating with RTP. For each participant, the session is defined by a particular pair of destination transport addresses (one network address plus a port pair—one for RTP and the other for RTCP). In a multimedia session, each medium is carried in a separate RTP session with its own RTCP packets. The multiple RTP sessions are distinguished by different port number pairs and/or different multicast addresses.

One method of transporting media streams, which applies to video delivered over IP, is sending streams that are logically multiplexed at the IP layer to multiple clients over the IP network. Typical manipulations in an IP network include: ■ ■

■

2

A new RTSP session is initiated and established between a new client and the server. RTP or UDP streams associated with that RTSP session are streamed (or added to the logical IP multiplex) by the Video on Demand (VOD server). Control commands similar to VCR controls are received and executed by the server on the outgoing media streams.

Java Stream Assembly Programmer’s Guide • December 2004

The compression formats for media streams can be any of the available open-standards formats. Common compression schemes are MPEG-2 and MPEG-4. Table 1 gives some idea of the typical bandwidth, in bits per second (BPS), required for transport media streams of various types.

TABLE 1

Comparison of Bandwidth Requirements

Content

Typical Range

Audio

28-400 Kbps

Interactive TV

100-500 Kbps

Internet-based Streaming Video

28 Kbps-1.5Mbps

Broadcast Television

3-4 Mbps

Teletext (full page)

100Kbps-4.5 Mbps

HDTV

10-60+ Mbps

In practice, the bandwidth of any media stream is limited by the maximum throughput of the physical link. For example, the maximum throughput of an Asynchronous Serial Interface (ASI) is 270 Mbps, and the maximum throughput of a Gigabit Ethernet link is approximately 1,000 Mbps. Two challenges must be overcome before servers and intermediate equipment can handle media streams effectively. ■

■

The first challenge is to process and transport the huge volumes of data contained in the media streams, where processing means to manipulate the media stream as it moves through a component. A multiplexer is a type of processor that combines media streams. It is a particularly significant type because multiplexing is central to video delivery. The second challenge—which compounds the first one—is that all the data in a media stream must be processed in real time, since any data that does not arrive on time is essentially worthless.

Typical stream-handling functions include the following: ■

■

■ ■ ■

Combining audio, video, interactive, and other data to create a Single Program Multiplex (SPTS or RTP channels for a single RTSP session). Inserting new data, such as advertising or metadata, into individual program transport streams. Combining SPTSs to create a multiple program multiplex (for example, an MPTS). Removing a program from a multiple program multiplex. Inserting a program into a multiple program multiplex (for example, dropping a service in MPTS or initiating an RTSP session).

Because of the tremendous volume of data being transported, an important design consideration for video servers is the avoidance of unnecessary copies of the same media stream data. That is, when moving data from an input port to an output port, data that exists at one level (for example, the device level) should not be unnecessarily copied to another level (for example, the operating system level). The Java Stream Assembly API

Chapter 1

Introduction to Assembling Media Streams

3

addresses the issue of extra data copies by providing multiple methodologies for transporting data. For example, it Java Stream Assembly supports intra-process shared memory to reduce stream copying.

Assembling Media Streams Assembly is the process of combining media streams to create a multiplex. Media multiplexers are hardware or software components that assemble media streams.

Media Delivery Mechanisms Two transport mechanisms are possible. The Java Stream Assembly API abstracts these transport mechanisms to handle both in a generic way, as shown in FIGURE 1.

FIGURE 1

Media Delivery Mechanisms

In one case, RTP/UDP and IP comprise the protocol and transport layers, respectively. In the other, MPEG-2 functions as both the protocol and transport layers. The Java Stream Assembly API supports both delivery mechanisms.

4

Java Stream Assembly Programmer’s Guide • December 2004

Controlling Multiplexers The control of multiplexers involves several procedures: ■

Starting and stopping media streams—This includes starting, stopping, pausing, and resuming the streams, as well as configuring, starting, and resetting the multiplexer.

■

Modifying the data within media streams—This includes processing the data to add and drop services, adding interactive data, adding special data like Teletext, and mapping services from one input to another.

■

Monitoring the performance of the multiplexer—Applications that use the Java Stream Assembly API can monitor the performance of the multiplexer. For example, the API provides access to transport stream overflow and underflow events, which are indicators of congestion.

Feeding Data to Multiplexers Multiplexers typically consume data at high bit rates. It is normal to have data either “pushed” to the multiplexer or “pulled” from the multiplexer at constant or variable rates. The source of the data can be a broadcast stream, IP stream, or local or remote file source. When real-time data is made available to the multiplexer, it is important to meet deadlines to ensure that the multiplexer is neither overloaded nor starved. Due to the sheer volume of data that a multiplexer can consume, it is typical to support different modes for moving data in and out of the multiplexer. The Java Stream Assembly API supports the most common mode, including shared memory segments to avoid unnecessary data copies. Further details are available in Section “Ports” on page 10.

Chapter 1

Introduction to Assembling Media Streams

5

6

Java Stream Assembly Programmer’s Guide • December 2004

CHAPTER

2

Java Stream Assembly API Overview This chapter provides an overview of the JavaTM Steam Assembly API. It includes the following sections: ■

Purpose of the Java Stream Assembly API

■

Design Requirements

■

Usage Scenarios

■

Fundamental Concepts

■

Applications and Components

■

Packages

Purpose of the Java Stream Assembly API The Java Stream Assembly API provides a standard interface for controlling the hardware and software components used to assemble and deliver multiplexed media streams that contain broadcast video, on-demand video, interactive media, and other time-critical content. Using this API, software developers can write Java applications to control and monitor any video server or other media stream assembly system or component that supports the Java Stream Assembly API. The Java Stream environment consists of media stream inputs (sources), stream processors that manipulate the media stream, and outputs (sinks). Sources and sinks may connect directly to an ASI or Gigabit Ethernet interface, or to some higher-level resource such as a network or file system. A common configuration consists of multiple sources, at least one processor, and at least one sink, residing on a single server. Other configurations are possible, however, including distributed architectures.

7

Design Requirements The Java Stream Assembly API was designed to meet the following requirements: ■ ■

■ ■

Must provide the ability to discover and configure multiplexers. Must provide control functions for assembling, starting, stopping, adding, and dropping streams. Must support both MPEG2 transport and IP streams. Must support the monitoring and reporting of stream events, such as stream underflow/overflow events.

Usage Scenarios ■

Broadcast Video—Sending the same stream to many different subscribers simultaneously.

■

On-demand Video—Sending different streams to different subscribers, as requested in real time by subscribers. Also called Video On Demand (VOD).

■

Interactive Television—Sending subscribers executable programs that are executed in a set-top box and displayed on a television. These programs offer some degree of interactivity, such as switching between pages of information (for example, between a five-day extended weather forecast page and a Doppler weather radar page). Also called iTV.

Fundamental Concepts This section introduces the following important concepts in the context of the Java Stream Assembly API:

8

■

Streams

■

Ports

■

Components, including: ■

Stream Sources and Stream Sinks

■

Processors

Java Stream Assembly Programmer’s Guide • December 2004

Streams As described in the “Media Streams” section of Chapter 1, a media stream is a continuing, time-critical flow of audio, video, and data. In the context of the Java Stream Assembly API, however, the term stream means a media stream flowing between the output port (source port) of any component and the input port (sink port) of any other component, as shown in FIGURE 2.

FIGURE 2

Relationship of Streams to Source and Sink Ports

An MPEG-2 elementary stream is a stream of packets of a single element, which can be video, audio, or other data. These elementary stream packets are logical blocks of compressed data of varying sizes. The size and type of logical blocks depend on the stream. For instance, while typical elementary stream packets for video are pictures, for audio they are frame units. These variably-sized blocks of elementary stream data are further packetized into units of constant size in an MPEG-2 transport stream. These constant-size units of 188 bytes are called MPEG-2 transport stream (TS) packets. TS packets of various elementary streams that belong to a single program are multiplexed together in a single-program MPEG-2 transport stream (SPTS). TS packets of each elementary stream carry a unique identifier called a Packet ID (PID). The Program Map Table (PMT) uniquely identifies the elementary streams and their PIDs. When an MPEG-2 transport stream carries elementary streams belonging to multiple programs, it is called a multi-program transport stream (MPTS). The Program Association Table (PAT) gives the association between programs and the constituent streams. For an SPTS, the PAT identifies only one program.

Chapter 2

Java Stream Assembly API Overview

9

The Java Stream Assembly API supports PATs and PMTs in detail with interfaces to synthesize and analyze them. TABLE 2 describes these interfaces. The Java Stream Assembly APIs must carry other program-specific and service-specific tables as well, but these are not supported in detail.

TABLE 2

MPEG-2 Table Controls

Class

Table Manipulation Function

javax.mediastream.format.MPEG2Component

Provides the information required to identify an elementary stream.

javax.mediastream.format.MPEG2StringDescriptor

Provides an implementation of the MPEG-2 string descriptor arrays.

javax.mediastream.format.PAT

Identifies MPEG-2 programs in a transport stream.

javax.mediastream.format.PIDMappingPair

Reads and writes the 'to' and 'from' the Program IDs (PIDs). A set of 'to' and 'from' values is called a 'mapping pair'.

javax.mediastream.format.PMT

Provides an abstraction of the Program Map Table defined in MPEG-2 systems. Provides read/write access to individual table fields.

Ports A port is a logical node that represents the point at which a stream enters or leaves a Java Stream Assembly component. Because components can be either physical devices or processes, a port does not always correspond to a physical interface. Ports can be either source ports or sink ports, where “source” and “sink” describe the relationship of the port to the stream, rather than to the component. Thus, the port associated with the component at which a stream originates is the source port for that stream, and the port associated with the component at which the stream terminates is the sink port. A source port is therefore always the output of some component, and a sink port is always the input to some component. In addition to being classified as sources or sinks according to their relationships to streams, ports are also classified according to the programming model used to move data between them (see “Moving Data Between Components: Port Types” on page 19 for more information). The three data-movement models, or port types, are: ■

■

■

10

Local Model—Used for moving data between components running on or associated with the same machine using shared memory segments. Directed Model—Used for moving low bit-rate data between components through the Java layer, this data model allows you to use Java calls to manipulate the data. Remote Model—Typically used for moving data between components that are networked together but not running on or associated with the same machine, this data model uses TCP/IP or some other networking protocol.

Java Stream Assembly Programmer’s Guide • December 2004

The source and sink ports at opposite ends of a stream must use the same datamovement model (see “Matching Port Types” on page 20 for more information).

Components The Java Stream Assembly API defines three main types of components (shown with bold borders in FIGURE 3): ■

■

■

Components that have one or more input ports and one or more output ports are called processors (StreamProcessor). Processors manipulate media stream elements via the InputMediaProcess interface and route streams from input ports to output ports. Components that have one or more source ports and no sink ports are called stream sources (StreamSource). Stream sources can be ASI cables, network devices, computer file systems, or storage devices. Components that have one or more sink ports and no source ports are called stream sinks (StreamSink). Stream sinks, like stream sources, can be ASI cables, network devices, computer file systems, or storage devices.

FIGURE 3

Component Hierarchy

Of course, some stream sink components have physical outputs of their own, just as some stream source components do. However, stream sources and stream sinks are defined as components with either source or sink ports, but not both. For the purpose of the Java Stream Assembly API, ports are defined as “logical nodes that define the two ends of a stream,” and streams are defined as “media streams traveling between the output port (source port) of any component within the scope of the Java Stream Assembly API and the input port (sink port) of any other component within the scope of the Java Stream Assembly API.” As shown in FIGURE 2 on page 9, the scope of the Java Stream Assembly API includes only the inner terminus of stream source and stream sink components. As a result of these definitions, stream sources might have input ports, but those input ports, if they exist, are outside the scope of the Java Stream Assembly API. Likewise, stream sinks might have output ports, but those output ports, if they exist, are outside the scope of the Java Stream Assembly API.

Chapter 2

Java Stream Assembly API Overview

11

Stream Sources and Stream Sinks In the context of the Java Stream Assembly API, the terms source and sink are used in two different contexts: Stream Source/Stream Sink—Components that have one or more source ports and one or more sink ports are called processors; they process a stream and pass it on, all under the control of the Java Stream Assembly API. In contrast, components that expose to the Java Stream Assembly API only source ports or sink ports are called either stream sources or stream sinks. Therefore, from the point of view of the Java Stream Assembly API: ■ ■

all of the ports in a stream source must be source ports, and all of the ports in a stream sink must be sink ports.

A file source is a special type of stream source, and a file sink is a special kind of stream sink. File sources and sinks are associated with files rather than media stream transport interfaces such as ASI. FIGURE 4 shows a stream source and stream sink in relation to a chain of streams. Note that while the stream source and stream sink might have other physical or logical interfaces than the ones shown, those other interfaces are outside the scope of the Java Stream Assembly API.

FIGURE 4

Stream Source and Stream Sink

Processors As defined by the Java Stream Assembly API, a processor is an intermediate component in a chain of streams managed by the Java Stream Assembly API. A processor has one or more source ports and one or more sink ports. A processor performs some operation on the stream and passes it on, all under the control of the Java Stream Assembly API.

12

Java Stream Assembly Programmer’s Guide • December 2004

The operation performed by a processor can be simply a pass-through, connecting a stream source to a stream sink. More often, however, the operation involves some manipulation of the stream—for example, adding or dropping specific media streams from multiplexes, or remapping certain PIDs to prevent collisions between identically named packets from different sources that are being merged.

Stream Control The stream control process is modeled as a state machine, as shown in FIGURE 5.

FIGURE 5

Stream Control State Diagram

Chapter 2

Java Stream Assembly API Overview

13

Transitions from state to state are triggered by invoking one of five methods: init(), prime(), start(), stop(), pause(), or resume(). Not all methods are legal from any given state. FIGURE 5 shows all possible transitions with the exception of those associated with the reset() method, which are omitted to avoid clutter. The reset() method can be invoked from any state, and forces an immediate transition to the Initialized state; if the state machine is already in the Initialized state, it remains in that state. TABLE 3 presents another view of the state machine. Note that if no transition is defined for an invoked method in the state machine’s current state, the InvalidStateException is generated, and the state remains unchanged.

TABLE 3 State

Condition

Action

Loaded

init()

Transition to Initialized state

reset()

Transition to Initialized state

any other

InvalidStateException

prime()

Transition to Primed state

start()

Transition to Started state

init()

Remains in Initialized state

reset()

Remains in Initialized state

any other

InvalidStateException

start()

Transition to Started state

stop()

Transition to Stopped state

prime()

Remains in Primed state

reset()

Transition to Initialized state

any other

InvalidStateException

stop()

Transition to Stopped state

pause()

Transition to Paused state

reset()

Transition to Initialized state

any other

InvalidStateException

resume()

Transition to Started state

stop()

Transition to Stopped state

pause()

Remains in Paused state

reset()

Transition to Initialized state

any other

InvalidStateException

start()

Transition to Started state

Initialized

Primed

Started

Paused

Stopped

14

Valid Transitions from Each State

Java Stream Assembly Programmer’s Guide • December 2004

TABLE 3 State

Valid Transitions from Each State Condition

Action

stop()

Remains in Stopped state

reset()

Transition to Initialized state

any other

InvalidStateException

There are two kinds of stream control attributes: control type and position type: ■

Control Type specifies what you want to do to the stream: start it, stop it, pause it, resume it, preload it, or drain it. See the “Control Types” section, below, for details.

■

Position Type specifies where in the stream you want the control to be executed. See the “Position Types” section, below, for details.

Control Types Four of the five control types—start(), stop(), pause(), and resume()—are similar to the equivalent controls on a VCR (subject to the limitations of the media type being viewed, as well as to the constraints described below). The fifth control type, prime(), is not analogous to any VCR control. The realizable precision of the control arguments depends on the type of media stream being controlled. The position specified by the application may actually differ from the position realized by the implementation, for three reasons. First, since the media samples are discrete, the position value specified by the application might fall between realizable position values. If the stream is video, for example, the argument might be positioned between video frames. The implementation in this case has no choice but to round to an adjacent, realizable value. Second, compression techniques often result in video samples that are independent of other media samples, while other video samples, such as interpolation frames, require adjacent frames to reconstruct. The media samples that are independent can be thought of as random access points. The implementation of such compression techniques might therefore round the client position value to an adjacent random access sample, because it cannot reconstruct media samples adjacent to the client position value. Finally, the stream might be live. For a live stream, if the client position value is in the past (less than the current stream position), the implementation rounds the value to the first realizable value after the current position and performs the control operation. If the client position value is in the future (greater than the current stream position), the implementation defers the control operation until the stream advances to that position. If there are otherwise no exceptions, the control operation returns, but the state machine does not transition until the stream advances to the client position value. The implementation then reports the transition through one of the operations on the ControlContext interface.

Chapter 2

Java Stream Assembly API Overview

15

The realizable Scale values can also differ from the scale that the client includes as an argument. If the stream is live, for example, the realizable Scale value is often 1.0. See the individual control descriptions in TABLE 4 for details.

TABLE 4

Descriptions of the Control Types

Control Type

Description

start()

This control starts the media stream from the beginning or, if the stream is live, at the first realizable position.

stop()

Because a stream’s source can be either live or storage, stop() does not always behave like the equivalent control on a VCR. For a live stream, if the client position value is less than or equal to the current value, the implementation stops the stream. While the source continues to forward the live stream and the stream position advances, the implementation does not need to journal the live stream to storage. This means that if the client later invokes start(position, scale), the same position as the previous stop(position), the implementation might not be able to reconstruct the portion of the stream between the previous stop() position and the current start() position. In a situation where the position of the start() is less that the current stream position, the implementation rounds the position to the current stream position (or to the first random access point after the current stream position).

pause()

16

If the state machine transitions to and remains in the paused state, the implementation guarantees that it will preserve the session/execution context at the time the client invokes pause(). This control contrasts with stop(), whose implementation does not guarantee retention of the session/execution context at the time that the client invoked the control. The implementation will resume() at the same position, assuming it is realizable as described above. If the implementation suspects it might lose resources, it should checkpoint the resource state such that after re-acquisition, it can return the resource to the previous context.

Java Stream Assembly Programmer’s Guide • December 2004

TABLE 4 Control Type

Descriptions of the Control Types Description

As an example, imagine a live stream for which the implementation elects to not journal the stream to storage. When the client invokes pause() and the current state of the state machine is started, the operation returns, but the state machine transitions to the Paused state and then immediately transitions to the Stopped state. The second transition signals the client that the implementation cannot necessarily resume() at the position of the previous pause() operation. resume()

This operation restarts the media stream at the position of the previous pause() operation, assuming that the pause() operation was successful. If the pause() operation was not successful and the state machine transitioned to the Stopped state, the resume() operation functions as start().

prime()

The syntax for this operation is prime(Position aPosition, Scale aScale). Prime() serves two purposes. First, the client can explore realizable values through the prime() operation. The implementation does not transport the stream after the pause(), but it does select the realizable Position and Scale. For example, if the stream is live and the implementation elects to not journal the stream to storage, the implementation might conclude that the realizable Scale is 1.0 and that the realizable Position is the current stream position. If the realizable values matter to the client, it invokes getScale() and getPosition()to learn the extent to which the implementation approximates the arguments. Additionally, prime() is a hint to the implementation to allocate resources such as buffers, and to forward the first media samples to them. These resource optimizations are not normative, and the implementation can elect to defer the initializations until later—for example, because allocation of resources at a given time would disrupt another stream. However, prime() does let the implementation prepare for subsequent start() invocations.

Note – The minimum conformance criteria require that the implementation support the immediate stream control operations, meaning that it must implement the full state machine. However, if it cannot preserve execution context (such as for a live stream), the pause() operation may cause a transition to the Paused state, followed immediately by a transition to the Stopped state.

Chapter 2

Java Stream Assembly API Overview

17

Position Types TABLE 5 describes the five position types.

TABLE 5

Descriptions of Position Types

Position

The Position class is a generic base class that is silent on the metric for position values. Because signatures can contain the base class, while the argument is a subclass, the Position type minimizes the class explosion that would result if the signatures contained each subclass.

Immediate

This position type executes the stream control operation at the first realizable position.

Clock

This position type executes the stream control at the specified time, using the time value found in the stream. For video streams, the standard practice is to place a program clock reference into the stream, whose value is often discontinuous in practice. To compensate, there is also a standard for normal play time that begins at the start of a stream with a value of zero and advances at the stream transport rate. (The transport rate need not equate to a scale value of one; for example, multiple versions of the stream might be available in storage with scale values greater than or less than one.) While the normal play time of content after its creation should be continuous, subsequent compositions of multiple stream sources can introduce multiple time lines that make this clock discontinuous.

Sample

The position type executes a stream control using video frames, audio samples, or other temporal units to identify a stream position. For most media streams, the sample rate is constant, so knowledge of the sample count allows the client to calculate the stream position as a time value.

Packet

This position type executes the stream control at the specified packet number. It is often difficult for the implementation to inspect inside the stream and recover time values or sample values. The alternative is for the implementation to count packets. For some media streams, the packet count approximates the stream time. For example, audio streams are often of a constant sample size and constant sample rate. If an audio passage (brief collection of contiguous samples) is placed into a packet of constant size, then the packet count becomes a surrogate for the stream time. For other media streams, such as video, the sample size (in this case, the video frame after compression) is variable. Thus, unless the client knows the stream both before and after compression, the packet count is at best a crude measure of stream time.

18

Java Stream Assembly Programmer’s Guide • December 2004

Physical Interfaces Stream sources and stream sinks can be of different types. For example, an ASI link, a Gigabit Ethernet link, or a file can act as either a stream source or a stream sink. The Java Stream Assembly API abstracts these different types so that the intermediate JsaComponents do not need to know the exact nature of the source and sink components to which they are connected.

Moving Data Between Components: Port Types The Java Stream Assembly API defines three standard port types for moving data between components—Local, Remote, and Directed—along with one optional, user-defined port type—ComponentInternal: Local Port Type—In this port type, the data moves directly between two components that are running or associated with the same machine through a shared or private memory segment. A typical arrangement involves a ring buffer that is accessed directly by the two components. Directed Port Type—In this port type, data moves through the Java layer. This gives a great deal of programmatic control for nonstandard manipulation of the media stream. It is only practical, however, where bit rates are relatively low. Remote Port Type—In this port type, components that are not co-located communicate over the data network using TCP/IP or some other networking protocol. ComponentInternal—Using this optional, user-defined port type, vendors can define vendor-specific implementations that may be viewed as aggregated. For example, an ASI card that also has a multiplexer may have a dedicated low-level means of connection that may be either hardware-assisted or inefficient to map into one of the models above. This approach is discouraged as a general-purpose means of connecting components because it does not promote interoperability. For a more detailed discussion of the above data movement models, see “Establishing a Stream Between Two JsaComponents” on page 29 of Chapter 3.

When to Use Each Port Type The Local port type can be used only when both components are running on or associated with the same machine, and when an appropriate buffer is available. The Remote port type can be used any time the components are not co-located. The Directed port type is an alternative to the Local one and should be used only after considering several factors, including the following: ■

■ ■

Does some factor, such as a nonstandard processing task, preclude using the Local port type? Can the data be processed fast enough to meet deadlines? Will the processing demands degrade system performance?

Chapter 2

Java Stream Assembly API Overview

19

Matching Port Types In the context of the Java Stream Assembly API, a stream can only exist between the source port of one component and the sink port of another component. When moving data, use one of the four port types defined in the Java Stream Assembly API—Local, Remote, Directed and ComponentInternal. In addition, the source and sink ports at opposite ends of a stream must use the same port type. See the previous section, “Moving Data Between Components: Port Types,” for more information on the four defined port types.

Applications and Components The Java Stream Assembly API can be modeled as a set of three tiers, as shown in FIGURE 6.

FIGURE 6 ■

■

■

20

Java Stream Assembly API Tiers

Client Tier—Java applications in this tier configure, manage, and monitor the complete Java Stream Assembly API system. Throughout this programmer’s guide, developers working in the client tier are said to be developing applications. The information in Chapter 3 is intended for application developers. Integration Tier—Java or C applications in this tier connect and manage streams between components, and provide a publish/subscribe event model to support the run-time monitoring of system activity. Throughout this programmer’s guide, developers working in the integration tier are said to be developing components. The information in Chapter 4 is intended for component developers. Resource Tier—Java or C applications in this tier control core source, sink, and processing hardware elements. Resource elements are controlled through their abstractions in the integration tier. Throughout this programmer’s guide, developers working in the resource tier are said to be developing components. The information in Chapter 4 is intended for component developers.

Java Stream Assembly Programmer’s Guide • December 2004

Roles and Responsibilities The Java Stream Assembly API consists of over 100 specified Java interfaces and classes organized within several packages. About 30% of this total are classes. Responsibility for implementing these classes and interfaces is divided among three roles: the platform provider, the component provider and the client developer.

Platform Provider Most of the Java Stream Assembly classes will be implemented by a platform provider. There are a handful of important classes provided which underpin the Java Stream Assembly architecture. These are the SessionManager, Session, Stream and StreamSet classes. These are of most use to a Client developer, but an understanding of them is essential. There are a several sorts of support classes such as permissions, positions and connection details that are used by components.

SessionManager This SessionManager class follows a singleton design pattern. This manager is used to create and track Session objects.

Session The Session object is used to create streams which are then associated with the session and may then be controlled by operations on the session. This class also provides the facility to create and notify SessionEvent listeners.

Stream The term stream refers to a media flow between the output Port (source port) of any component and the input Port (sink port) of another component. A Stream object is responsible for connecting the components, selecting a control interface, and monitoring control events. The Stream class makes many calls into a component.

StreamSet A StreamSet is a collection of streams. This allows several streams to be controlled as an aggregate.

Chapter 2

Java Stream Assembly API Overview

21

Component Provider The component provider will need to provide implementing classes for most of the Java Stream Assembly interfaces. They will be implementing associated interfaces for either an Endpoint or StreamProcessor. Some of the methods they will implement are needed by the platform. Others are called by a client application. There are several key interfaces: JSAComponent, ControlNode, Port and Control. Most of these interfaces follow a factory method design pattern.

JSAComponentFactory This interface provides a means of creating instances of JsaComponent(s) from a given vendor based on a set of attributes or properties.

JSAComponent This is the base interface for all realized components. This provides methods to create associated ControlNodes. There are many types of JSAComponents. They all fall under two subinterfaces: StreamProcessor or Endpoint. These interfaces themselves have subinterfaces, which typically are the component that the component provider is implementing (for example, StreamNetSource or MPEG2Assembler)

StreamProcessor A StreamProcessor extends the JSAComponent interface. There are methods to add and remove routes (cross-bar connection) between ControlNodes associated with Ports on the component. A specific realization of a StreamProcessor (for example, MPEG2Assembler) will also have specific methods to implement.

Endpoint An Endpoint also extends the JSAComponent interface. An EndPoint represents a media source or sink in a chain of JSAComponents -- there are StreamSource and StreamSink subinterfaces. Ultimately the Endpoint to be implemented is itself a subinterface of StreamSource or StreamSink. A few examples of these include: ■ ■ ■

Files (StreamFileSource, StreamFileSink) ASI ports (StreamASISource, StreamASISink) Network ports (StreamNetSource, StreamNetSink)

Each of these possesses methods unique to the type of Endpoint component.

22

Java Stream Assembly Programmer’s Guide • December 2004

ControlNode A ControlNode is associated with a Ports on a component. A ControlNode is a central point of command and control for ports. It provides methods for constructing and connecting Ports, obtaining Control interfaces and registering for events using the EventManager interface.

PortFactory The PortFactory interface has methods to provide a list of supported PortTypes, to create Port instances of a given PortType. It also has some navigation methods to return associated Ports and ControlNodes.

Port Port is the base interface for all communications between JSAComponents. Ports represent the points at which a Stream enters or leaves a Component. There are methods to open, close, start and stop. Ports have a PortType attribute which is classified by the programming model used to move data between them. These are well described in “Ports” on page 10 of this chapter and in Chapter 4, “Developing Components, as well as in the API documentation. The Component provider must choose one or more PortTypes which match the Component they are implementing. The type of Component being provided and the system architecture often dictate these choices. ■ ■ ■

PortTypes are either Sinks or Sources. PortTypes support the Push or Pull model. The Component provider must implement the underlying communications mechanism corresponding to the PrimaryType attribute of the Port (ComponentInternal, Directed, Local or Remote). This mechanism is used to communicate with peer Ports on other components.

These other peer Ports may have been implemented by other component providers. Finally one of the two connected ports must provide ConnectionDetails. Often this will be the “master” port (either a PushSource or PullSink).

ConnectionCtrl/ConnectionDetails A Port extends the ConnectionCtrl interface. This interface is used by the platform in the process of connecting Ports. It provides a methods to obtain ConnectionDetails from the ConnectionDetails provider port and given this object, complete a connection to the peer Port.

Chapter 2

Java Stream Assembly API Overview

23

Control Factory The selection process for a Control implementation is characterized in this interface.

Control The Control interface allows the application to direct the advance of a media stream (equivalent to the function of the controls on a VCR player). The Control interface implements a StreamControl state machine (see the API documentation and “Stream Control” on page 13 in this chapter for details). The base Control interface specifies a positionless control mechanism. There are methods such as start(), stop(), pause(), resume() and prime(). The PositionControl interface specifies where in the stream the you want the control to be executed. The Position class lists the various position types that can be supported.

Client Developer A few of the Java Stream Assembly interfaces are expected to be provided by the Client developer. The Client developer’s main job may be to assemble components and platform and provide an application. One application responsibility is to instantiate components and plumb them together. Another application responsibility is to provide monitoring and control.

ControlContext This interface provides the client-side context through which the Control reports state transitions. The design pattern accounts for two features of stream control: ■

■

Control operations can take considerable time, and thus are best cast as requests with subsequent confirmations, Spontaneous state transitions can occur, such as when the Control detects the EndOfStream condition.

EventListeners The application can register with the EventManager interface to receive a host of events generated by various components.

24

Java Stream Assembly Programmer’s Guide • December 2004

Packages Packages include the following: javax.mediastream.component—Includes the base classes and interfaces for all components, component factories, and control nodes. This package also includes related Permission classes and exceptions. javax.mediastream.control—Contains base classes for control interface and control callbacks (ControlContext). This package also includes related Permission classes and exceptions. javax.mediastream.control.factory—Contains only a factory interface for creating control interfaces. javax.mediastream.control.position—Includes the base interfaces for position-based control and related callbacks (PositionControlContext). javax.mediastream.endpoint—Contains generic interfaces representing entry and exit points from a complete stream assembly. javax.mediastream.events—Contains interfaces for registration of event listeners and the event classes themselves. javax.mediastream.format—Contains MPEG-2 and SDP related format classes and interfaces. javax.mediastream.port—Contains base classes and interfaces for ports. Includes port factories, connection control, push and pull sources, and sinks. javax.mediastream.processor—Contains base classes and interfaces for IP and MPEG based processors. javax.mediastream.stream—Contains stream and session related classes and interfaces. Also includes Permission classes for session management and related exceptions. javax.mediastream.stream.events—Contains stream events that may be published to each stream in a given session. Currently the only concrete event type is an RTCP event.

Chapter 2

Java Stream Assembly API Overview

25

26

Java Stream Assembly Programmer’s Guide • December 2004

CHAPTER

3

Developing Applications For the purposes of this programmer’s guide, the term application refers to an application that runs at the service provider’s premises and that consumes services provided by the JavaTM Stream Assembly API and the components that implement the Java Stream Assembly API’s interfaces. The term client refers to the receiver at the customer’s premises. This chapter contains the following sections: ■

Common Development Tasks

■

Delivering Broadcast Video

■

Delivering On-Demand Video

■

Delivering Interactive TV

Common Development Tasks The application developer is responsible for certain tasks. See “Roles and Responsibilities” on page 21 for details. Regardless of the specific application requirements, an application client developer often needs to perform a number of common tasks. These include: ■

■ ■

■ ■ ■ ■ ■

Looking up a JsaComponent in a naming service. ■ Creating a JsaComponent instance given a JsaComponentFactory. Looking up a JsaComponentFactory. Establishing a stream between two JsaComponents. ■ Establishing stream connection using the default connection strategy. ■ Establishing a stream connection using a given strategy. ■ Establishing a stream connection using a specific port type. Using port types directly. Controlling a stream flow. Monitoring stream flow events. Monitoring “bit-rate out of range” events. Monitoring over-run and under-run events.

27

Creating a Session Sessions are used to keep track of a collection of streams that share a common life cycle. For example, in the case of a stream assembly that is being used to provide video and audio through RTP, a number of streams relate to a single user session. The Session object can then be used to propagate RTCP events to all streams participating in that user session. The Session object also provides a means of shutting down the collection of streams when the user session terminates. The SessionManager provides a means of obtaining all active sessions, and therefore all active streams. This makes it possible to gain a complete picture of all running streams and components. CODE EXAMPLE 1 Creating a Session Session session = SessionManager.getInstance().createSession();

// Creating a collection of three related streams in this session Stream stream1 = session.createStream(); Stream stream2 = session.createStream(); Stream stream3 = session.createStream();

Looking Up a JsaComponent The Java Stream Assembly API relies on the standard Java Naming and Directory Interface (JNDI) API for component lookup and discovery. JNDI provides a consistent means of accessing a number of naming and directory services, including LDAP, COS Naming, and DNS. Of these, LDAP is the most likely choice for a Java Stream Assembly API environment. For details on JNDI, refer to http://java.sun.com/products/jndi/.

Why Use a Lookup Service? Components may be instantiated directly using the new operator only when the fully qualified name of the class is known. Each vendor provides its own implementation with a different name, however, causing potential lookup problems. Thus, although direct instantiation can be used, it does not support the notion of application portability and component interoperability. Factory classes solve this problem by providing a means of creating an object instance without knowing the fully qualified class name. However, where the factory class is also a vendor-specific implementation, the problem still exists. Naming services solve the object creation problem by providing a means of obtaining an object instance by name, allowing you to obtain a single instance of a vendor-specific factory. This factory can then be used to create instances of the object without knowing the vendor-specific class name.

28

Java Stream Assembly Programmer’s Guide • December 2004

Looking Up a JsaComponent When the administrator has configured and registered a component, an application may retrieve the object by name. Such components are known as administered objects, meaning that they are pre-configured and ready to be used by the application. CODE EXAMPLE 2 Looking Up a JsaComponent // Create initial lookup context. InitialContext ctx = new InitialContext();

// Look up the 'LiveFeed' by name JsaComponent factory = (JsaComponent)ctx.lookup("LiveFeed");

Looking Up a JsaComponentFactory In some cases, it is more appropriate to obtain an object factory, such as a file source, than it is to look up a JsaComponent (see the “Stream Sources and Stream Sinks” section in Chapter 2 for more information). Multiple file source instances may be created, each streaming different content and distinguishable only by their source file names. CODE EXAMPLE 3 Looking Up a JsaComponentFactory // Create initial lookup context. InitialContext ctx = new InitialContext();

// Lookup the 'FileStreamFactory' by name. JsaComponentFactory factory = (JsaComponentFactory)ctx.lookup("FileStreamFactory"); // Create an instance of the file source from factory. JsaComponent fileStreamComp = factory.createInstance(new Properties());

In other cases, an administered object approach might be better.The component factory scenario differs from an ASI source, in which the number of physical inputs—and therefore component instances tied to those inputs—is fixed.

Establishing a Stream Between Two JsaComponents “Ports” on page 10 described the differences between port types. This section explains how to code connections between components. The primary purpose of the Java Stream Assembly API is to configure, control, and monitor media streams. These streams flow between the ports on JsaComponents. The stream connection between two components is represented by the Stream object. Although the real stream does not flow through this object, the Stream object responsible for connecting the components, selecting a control interface, and monitoring control events.

Chapter 3

Developing Applications

29

Only ports that are compatible can be connected successfully. Automated selection mechanisms called strategies are available to manage port connections, but the Java Stream Assembly API allows application developers to take direct control if needed. A strategy specifies the order in which ports will be selected for connection under different circumstances. When executing a strategy, the platform automatically takes into consideration and handles all port compatibility issues. The Java Stream Assembly API includes a default strategy that optimizes performance in the most generic case. Application developers can create additional strategies to optimize performance under specific conditions. The following subsections deal with the different ways in which you can handle stream connections: ■ ■

Creating a Stream Connection Using the Default Strategy Creating a Stream Connection Using a Custom Strategy

Most developers will use the default strategy. However, in some components or applications, the optimal order of port selection may differ from that used in the default strategy. The following subsection contains instructions for creating a custom strategy if needed: ■

Creating a Custom Strategy

Creating a Stream Connection Using the Default Strategy Each component provides a list of the port types that it supports. A vendor might choose to provide an implementation of the Local (shared-memory) port along with the Remote (TCP/IP) port and omit the Directed (Java platform-enabled) port. Another vendor may choose to implement only the Remote (TCP/IP) port. When presented with two components to connect, the Stream object first creates a list of compatible pairs of ports from those provided by each component. (For a discussion of what constitutes a compatible pair, see the “Matching Port Types” section of Chapter 2, as well as in “Compatible Port Pairs” on page 33 of this chapter.) The list of compatible pairs is then searched in order of preference: ComponentInternal, Local, Remote, and Directed. The first match is returned as the port type to be used. These PortType objects are then passed back to the component as the argument to the port creation method. The two resulting ports are then connected. This is known as the default port connection strategy, or default strategy.

30

Java Stream Assembly Programmer’s Guide • December 2004

The following code example illustrates the default strategy: CODE EXAMPLE 4 Creating a Stream Connection Using the Default Strategy // Two components JsaComponent source; JsaComponent sink;

// The session Session jsaSession = SessionManager.getInstance().createSession(); // Create a stream Stream jsaStream = jsaSession.createStream(); // Connect using default strategy jsaStream.connect(source.createControlNode().getPortFactory(), sink.createControlNode().getPortFactory() );

Creating a Custom Strategy In situations where the default strategy does not suffice or is inappropriate, component providers can define a custom Strategy. A custom Strategy is developed by implementing the javax.mediastream.stream.Strategy interface. It has a single method called "SelectOption". SelectOption takes an array of compatible port type pairs as argument and returns the port pair selection that must be used for establishing the connection. In the sample code below, a custom strategy is created which is similar to the default strategy except that it favors a directed port over a remote port. CODE EXAMPLE 5 Creating a Custom Strategy public class CustomStrategy implements Strategy { public PortTypePair selectOption(PortTypePair[] portTypePairs) {

String[] types[0] types[1] types[2]

types = new String[3]; = PortType.LOCAL_RING_BUFFER_PORT; = PortType.DIRECTED_PORT; = PortType.REMOTE_TCP_PORT;

for (int i = 0; i < types.length; i++) { for (int j = 0; j < portTypePairs.length; j++) { PortTypePair pair = portTypePairs[j]; String primaryType = pair.getSourcePortType().getPrimaryType(); if (primaryType.equals(types[i])) { return pair; } } } return portTypePairs[0]; } }

Chapter 3

Developing Applications

31

Note – If you want to use a specific port type, create a custom strategy like the one shown above, but populated with just one port type.

Creating a Stream Connection Using a Custom Strategy Port type selection strategy can be user-defined. In this case, the preference order can be set by creating a stream with a list of port types in the required order of preference. The following code example shows how to creating a stream using the custom strategy created in the preceding example. CODE EXAMPLE 6 Creating a Stream Connection Using a Custom Strategy // Create a new Stream with given strategy Stream jsaStream = jsaSession.createStream(strat);

// Connect the JsaComponents using jsaStream.connect(source.createControlNode().getPortFactory(), sink.createControlNode().getPortFactory() );

PortType The PortType object describes each port in terms of four attributes: ■ ■

■

Source or Sink—A source port represents the source of a media stream. Streams flow from source to sink port. A stream connects a source port and a sink port. Push or Pull—A push source port pushes data from the source at the required rate for the media. It must be matched by a push sink port. A pull sink port pulls data from the source at the required rate for the media. It must be matched by a pull source port. Connection Details Provider or Connection Details User—The connection between two ports can be established by the source connecting to the sink, or vice versa. For example, in the case of a Remote (TCP/IP) port, the source could act as a TCP/IP server and wait for connection requests from the sink, or the sink could act as the server. In either case, once a connection is established, the stream can be pumped through the socket. When a PortType is set as a ConnectionDetails provider, it expects another port to connect to it. Therefore, it makes its connection details available to the other port. The ConnectionDetails object is a base class for all specific connection details. From the perspective of the stream or the application, nothing but the base class is ever visible. The platform obtains the ConnectionDetails from the provider and passes them on to the consumer in the connect call. The client port then initiates a connection to the provider port.

■

32

Primary Type—Defines a port as being Local, Directed, Remote, or ComponentInternal (see “Ports” on page 10 for details).

Java Stream Assembly Programmer’s Guide • December 2004

Compatible Port Pairs The stream implementation ensures that the ports being connected are compatible types. To do this, it calls a method ( isCompatibleWith) on the PortType that checks port compatibility by verifying that all of the following conditions are met: ■ ■ ■ ■

Both are of the same primary type, for example, Local. One is a source and the other is a sink. Both support a push model or both support a pull model. One port can provide connection details, while the other can use these details to create a connection.

In addition, if the primary type is Local (Ring Buffer), isCompatibleWith verifies that the implementations reside on the same physical hosts.

Controlling the Stream Flow The supported Control types are outlined in the “Control Types” section of Chapter 2. From the developer’s perspective, a single Control interface is required to control the stream. The application may also be interested in either spontaneous or controlled state transitions. The stream implementation provides the user with a single Control interface for the complete stream. The implementation delegates the user’s commands directly to the underlying ControlNodes. Although each ControlNode may support one or more Control types, they must all support the basic positionless Control type. The user may obtain a list of supported types and select the most appropriate one(s) for the stream.

Note – The list of available types returned to the user for the stream is the subset of types supported by both the source and sink.

Basic Stream Control The following example illustrates the use of the simplest available method of stream control. Because all ControlNodes must support at least this basic method, the type of control selected is set without checking the capabilities of the node. CODE EXAMPLE 7 Controlling the Stream in the Most Basic Manner // Two components JsaComponent source; JsaComponent sink;

// The session Session jsaSession = SessionManager.getInstance().createSession(); // Create a stream Stream jsaStream = jsaSession.createStream();

Chapter 3

Developing Applications

33

// Connect using default strategy jsaStream.connect(source.createControlNode().getPortFactory(), sink.createControlNode().getPortFactory() ); // obtain ControlFactory ControlFactory controlFactory = stream.getControlFactory(); // select basic Position Control type Control control = controlFactory.select(new Position()); // create dummy context - not defined here, see next section ControlContext controlContext = new DummyContext(); // set the control context control.init(controlContext); // now use control... // create default scale of 1/1 Scale scale = new Scale(); // start playing at normal speed. control.start(scale); // some time later... control.pause(); // some time later... control.resume(scale); // some time later... control.stop();

Position-based Stream Control The following example illustrates the use of a position-based stream control. CODE EXAMPLE 8 Using Position-Based Stream Control // Two components JsaComponent source; JsaComponent sink;

// The session Session jsaSession = SessionManager.getInstance().createSession(); // Create a stream Stream jsaStream = jsaSession.createStream(); // Connect using default strategy jsaStream.connect(source.createControlNode().getPortFactory(), sink.createControlNode().getPortFactory() );

34

Java Stream Assembly Programmer’s Guide • December 2004

// obtain ControlFactory ControlFactory controlFactory = stream.getControlFactory(); // get Set of supported positions Set positions = controlFactory.getPositions(); // check if the one I want is here if(!positions.contains(new ClockPosition())) { throw new Exception("No clock position control available"); } // select basic Position Control type PositionControl control = (PositionControl)controlFactory.select(new ClockPosition()); // create dummy context - not defined here, see previous section ControlContext controlContext = new DummyContext(); // set the control context control.init(controlContext); // now use control... // create default scale of 1/1 Scale scale = new Scale(); // create ClockPosition for posn in stream ClockPosition posn = new ClockPosition(123L); // start playing at normal speed. control.start(posn, scale); // some time later... control.pause(); // some time later... control.resume(scale); // some time later... control.stop();

Monitoring Stream Flow Events Because the state of the control might change spontaneously or might occur after a given command, it is useful to have some means of detecting state transitions. The user creates an implementation of ControlContext and passes it in to the control as part of the initialization sequence. A simple implementation of ControlContext might look like this:

Chapter 3

Developing Applications

35

CODE EXAMPLE 9 Implementing ControlContext class DummyContext implements ControlContext{

public java.util.Properties getProperties(){ return new Properties(); } public void cancelled(int state){ System.out.println("Cancelled"); } public void drained(){ System.out.println("drained"); } public void finished(){ System.out.println("finished"); } public void initialized(){ System.out.println("initialized"); } public void paused(){ System.out.println("paused"); } public void primed(Scale aScale){ System.out.println("primed with "+aScale); } public void resumed(Scale aScale){ System.out.println("resumed with"+aScale); } public void started(Scale aScale){ System.out.println("started with "+aScale); } public void stopped(){ System.out.println("stopped"); } public void stopped(java.lang.Throwable aThrowable){ System.out.println("stopped"); } }

36

Java Stream Assembly Programmer’s Guide • December 2004

Monitoring “Bit-rate Out of Range” Events The application may subscribe to “bit-rate out of range” events from a ControlNode. The ControlNode can have more than one subscriber, and each subscriber may determine its own unique definition of “out of range.” As part of the subscription, the application must include a BitRateOutOfRangeParams object that provides the upper and lower limit thresholds. CODE EXAMPLE 10 Providing a BitRateOutOfRangeParams Object // Two components JsaComponent source; JsaComponent sink;

// The session Session jsaSession = SessionManager.getInstance().createSession(); // Create a stream Stream jsaStream = jsaSession.createStream(); // Connect using default strategy jsaStream.connect(source.createControlNode().getPortFactory(), sink.createControlNode().getPortFactory() );

// get the EventManager for the source Control Node EventManager sourceEventManager = stream.getSourceControlNode().getEventManager(); // Create range object 4 Mbps to 1 Mbps BitRateOutOfRangeParams params = new BitRateOutOfRangeParams(4000000, 1000000); // create simple event handler BitRateOutOfRangeListener listener = new BitRateOutOfRangeListener(){ public void onEvent(BitRateOutOfRangeEvent e){ System.out.println(e); } }; // register listener with range paramaters sourceEventManager.addBitRateOutOfRangeListener(listener, params);

Monitoring Overflow and Underflow Events The application can subscribe to overflow and underflow events from a Control Node. An overflow occurs when a push source runs out of buffer space while attempting to send data to the sink. An underflow occurs when a pull sink finds that the buffer contains no data while attempting a read.

Chapter 3

Developing Applications

37

The following example illustrates overflow event monitoring. Underflow monitoring is identical, apart from the name change. CODE EXAMPLE 11 Monitoring an Overflow Event // Two components JsaComponent source; JsaComponent sink;

// The session Session jsaSession = SessionManager.getInstance().createSession(); // Create a stream Stream jsaStream = jsaSession.createStream(); // Connect using default strategy jsaStream.connect(source.createControlNode().getPortFactory(), sink.createControlNode().getPortFactory() ); // get the EventManager for the source Control Node EventManager sourceEventManager = stream.getSourceControlNode().getEventManager(); // create simple event handler OverflowListener listener = new OverflowListener() { public void onEvent(OverflowEvent e) { System.out.println(e); } }; // register listener sourceEventManager.addOverflowListener(listener);

Delivering Broadcast Video The scenario in this section illustrates one strategy for delivering broadcast video by completing the following tasks: ■

Combine a national television broadcast feed (Audio Visual ASI inputs, encoded off-site and originating at a terrestrial TV head-end) with a regional television feed (Audio Visual ASI inputs, encoded locally) to create a multiplex.

■

Provide correct signaling for the media streams in the new multiplex (by remapping PIDs as required to prevent duplication within the multi-program transport stream).

See the “Streams” on page 9 of Chapter 2 for background information on the Java Stream API classes used for modifying MPEG-2 table values.

38

Java Stream Assembly Programmer’s Guide • December 2004

Note – Each of the following subsections describes a task that a client handling broadcast TV transactions typically must accomplish. For the most part, these tasks are specific to broadcast TV transactions. See “Common Development Tasks” on page 27 of this chapter for a discussion of tasks that are common to more than one use case. FIGURE 7 is a block diagram of the Broadcast Video example.

FIGURE 7

Block Diagram: Broadcast Video Code Example

Initialization When working with broadcast video, the initialization sequence consists of the following steps: ■ ■ ■ ■ ■

Creating a Processor Instance Setting Up Output and Inputs Getting and Checking Media Attributes Creating a File Source Instance Connecting Sources and Sinks

The code examples that follow correspond to the steps listed above. FIGURE 8 shows the sequence diagram for the initialization phase.

Chapter 3

Developing Applications

39

FIGURE 8

40

Broadcast Video Initialization Sequence

Java Stream Assembly Programmer’s Guide • December 2004

The first code example shows how to create a processor instance. CODE EXAMPLE 12 Creating a Processor Instance // Get the processor factory from the initial context. InitialContext rootContext = new InitialContext();

// Create a processor instance from the processor factory. JsaComponentFactory processorFactory = (JsaComponentFactory)rootContext.lookup("MyProcessorFactory"); Attributes mpeg2AssemblerAttributes = new BasicAttributes(true); mpeg2AssemblerAttributes.put(new BasicAttribute("Name","MyMPEG2Assembler")); MPEG2Assembler myMpeg2Assembler =(MPEG2Assembler)processorFactory.createInstance(mpeg2Assembler Attributes);

The next code example shows how to create an output (ASISink) and input (ASISource). This involves several substeps: ■ Looking up the ASI output ■ Creating an ASISink instance for it ■ Assigning attributes to the ASISink instance ■ Looking up the ASI input (the live feed) ■ Creating an ASISource instance for it ■ Assigning attributes to the ASISource instance CODE EXAMPLE 13 Creating and Assigning Attributes to Outputs and Inputs // Look up ASI Output "Local ASIDistribution" // Create an ASISink instance from the ASI sink factory. JsaComponentFactory asiSnkFactory = (JsaComponentFactory)rootContext.lookup("My Vendor ASI Sink Factory"); Attributes asiSnkAttributes = new BasicAttributes(true); asiSnkAttributes.put(new BasicAttribute("Name", "Local ASIDistribution")); StreamASISink asiSink = (StreamASISink)asiSnkFactory.createInstance(asiSnkAttributes); // Lookup ASI Input "LiveFeed" // Create an ASISource instance from the ASI source factory. JsaComponentFactory asiSrcFactory = (JsaComponentFactory)rootContext.lookup("My Vendor ASI Source Factory"); Attributes asiSrcAttributes = new BasicAttributes(true); asiSrcAttributes.put(new BasicAttribute("Name", "Live ASI Feed")); StreamASISource asiSource = (StreamASISource)asiSrcFactory.createInstance(asiSrcAttributes) ;

Chapter 3

Developing Applications

41

The next code example shows how to determine whether the media attributes associated with the input stream (ASISource) are of the right type for the processor that was created earlier. CODE EXAMPLE 14 Getting and Checking Media Attributes MediaAttributes liveSrcMediaAttributes = asiSource.getMediaAttributes(); if (!(liveSrcMediaAttributes instanceof MPEG2Attributes)) { throw new IllegalArgumentException("Unsupported Media AttributesType"); }

The next code example shows how to open a local insertion file source that will provide the second stream input to the stream processor myMPEG2Assembler, which will multiplex them to generate the final broadcast stream sent to the ASISink. This involves several substeps: ■ ■ ■ ■

Looking up the file source input Creating a file source instance Assigning attributes to the file source Checking that the file source media type matches the processor type

CODE EXAMPLE 15 Creating and Opening a File Source Instance // Look up the file source input "Local ProgrammingFileSource" // Create a file source instance from the file source factory. JsaComponentFactory fileSourceFactory = (JsaComponentFactory)rootContext.lookup("My Vendor File Source Factory"); Attributes fileAttributes = new BasicAttributes(true); fileAttributes.put(new BasicAttribute("Name", "Local ProgrammingFileSource")); StreamFileSource fileSource = (StreamFileSource)fileSourceFactory.createInstance(fileAttribut es); // Open the filesource with filename "/LocalInsertionofToday" fileSource.open("/LocalInsertionofToday"); // Get MediaAttributes for content and check if it is of the // right type MediaAttributes localMediaAttributes = fileSource.getMediaAttributes(); if (!(localMediaAttributes instanceof MPEG2Attributes)) { throw new IllegalArgumentException("Unsupported Media Attributes Type"); }

42

Java Stream Assembly Programmer’s Guide • December 2004

The final code example for the initialization sequence shows how to connect the two sources (one file source and one ASI source) to the processor (MPEG2 assembler) and the processor to the sink (ASI sink). Note the use of a session object to create and manage the media streams. FIGURE 9 shows the sequence of operations that corresponds to code example 16. CODE EXAMPLE 16 Connecting Sources and Sinks // Create a session object Session session = SessionManager.getInstance().createSession();

// Connect fileSource and MPEG2 assembler Stream fsMuxStream = session.createStream(); ControlNode fsCn = fileSource.createControlNode(); fsMuxStream.connect(fsCn.getPortFactory(), myMpeg2Assember.createControlNode().getPortFactory() ); // Connect ASI source and MPEG2 assembler Stream asiSrcMuxStream = session.createStream(); ControlNode asiSrcCn = asiSource.createControlNode(); asiMuxStream.connect(asiSrcCn.getPortFactory(), myMpeg2Assember.createControlNode().getPortFactory()); // Connect ASI sink and MPEG2 assembler Stream asiSnkMuxStream = session.createStream(); asiSnkMuxStream.connect (asiSink.createControlNode().getPortFactory(), myMpeg2Assember.createControlNode().getPortFactory());

Chapter 3

Developing Applications

43

FIGURE 9

44

Broadcast Video Changes During Operation Sequence

Java Stream Assembly Programmer’s Guide • December 2004

Processing Streams The processing done in this example is typical of a broadcast video application: the processor inserts local content from a file source at the appropriate times into the live feed (an ASI stream) and passes the processed stream to the ASI sink for local broadcast. FIGURE 10 shows the sequence for processing broadcast video streams.

FIGURE 10

Broadcast Video Stream Processing Sequence

Chapter 3

Developing Applications

45

The code example for the processing sequence shows how to configure the processor (MPEG2 assembler) and verify that the ASI sink has sufficient bandwidth to handle the output of the processor. Processing Streams, Broadcast Video Example // Configure Mpeg2Assembler MyMpeg2Assembler.configureAutoPSI(true); MyMpeg2Assembler.configureRemapPIDs(true); MyMpeg2Assembler.configurePCRRestamp(true); MyMpeg2Assembler.configureForceNIT(asiSrcCn); MyMpeg2Assembler.setPSIInterval(100); MyMpeg2Assembler.setTransportStreamID(200); CODE EXAMPLE 17

InputMediaMPEG2Process asiSrcProcess = MyMpeg2Assembler.getInputMediaProcess(asiSrcCn); long outputBitRate = asiSrcProcess.getStreamBitRate(); InputMediaMPEG2Process fsProcess = MyMpeg2Assembler.getInputMediaProcess(fsCn); outputBitRate += fsProcess.getStreamBitRate(); outputBitRate += 100000; // Check bitrate on physical interface ASIout1 PhysicalInterface[] asiSinkIF = asiSink.getPhysicalInterfaces(); for( int i=0; i < asiSinkIF.length; i++) { if( asiSinkIF[i].getDeviceName().equals("ASIout1") ) { if( asiSinkID[i].getTransportBitRate() < outputBitRate ) { throw new ProcessorException("Sum of Input bit rates more than output bit rate"); } break; } }

Changes During Operation The Java Stream Assembly processing allows programmatic control of the streams and processors. Refer to the API documentation for a complete summary of the available controls, and to the documentation provided by component vendors for componentspecific controls. The purpose of this section is not to be exhaustive, but rather to demonstrate the general form for making changes to the stream processing during operation.

46

Java Stream Assembly Programmer’s Guide • December 2004

CODE EXAMPLE 18 Changes During Operation, Broadcast Video Example // Changing processing in an input InputMediaProcess fsProcess = MyMpeg2Assembler.getInputMediaProcess(fsCn); fsProcess.setSPTSProgramNumber(10);

// Deleting an input fsMuxStream.disconnect(); fsMuxStream.finalize(); filesource.releaseControlNode(fsCn);

Delivering On-Demand Video The code samples that follow are based on an installation that delivers video on demand (VOD) via an IP network. In this example, the subscriber selects a video to watch from a list of available videos, and the client device on the subscriber’s premises requests the video using a filename (such as tahiti.mpg4). Because this is a VOD example, each user selects video programming independently from other users. The media stream(s) associated with the video selection is delivered to the requestor over IP as a unique program in MPEG-2 or MPEG-4 format.

Note – Although MPEG-2 is used here as a compression format, the data delivered could also be an MPEG-2 transport stream containing other compression formats. A media stream in MPEG-2 TS or MPEG-4 format is delivered to the user’s set-top box or client over IP. The transport protocol can be raw UDP, RTP over UDP, or even TCP. The control protocol is RTSP. FIGURE 11 is a block diagram of the VOD example.

FIGURE 11

Block Diagram: VOD Example

Chapter 3

Developing Applications

47

Delivery Scenarios A stream in the IP format can be delivered through one of the following scenarios: ■

■

The content file, which is identified by content_name, is streamed to IPAddress, Port. The client also indicates the desired transport for receiving the data, such as RTP/UDP, UDP, or TCP. Live feed data is received over a socket in RTP or MPEG-2 TS and sent to a client in RTP form. If the data is received in MPEG-2 TS format, the processor packetizes the data into RTP.

The following processes occur during the streaming of MPEG4 content using the RTP transport protocol: Performing the start-up procedures: ■ ■

The system reads the configuration of available disks and network interfaces. The system initializes services such as database and delivery service.

Provisioning a video: ■

■ ■

The MPEG2 TS or MPEG4 video file is made available to the video server, along with metadata describing its content. The video server processes the content and generates any required auxiliary streams (such as a REW or FF stream) from the content. The original and auxiliary content is stored on disk, and the metadata is stored in a database.

Starting a stream: ■ ■

■

■

The user selects a video using a browse/selection UI. A set-top box or client acting on behalf of the user sends an RTSP request (consisting of DESCRIBE, followed by SETUP and PLAY) to the video server. The video server selects the storage source from which to read the video, along with the network interface to be used for transmission. The delivery service begins to read the video from the specified source and pump it over the specified network interface.

Switching to FF/REW mode: ■ ■

The set-top box or client sends an RTSP PAUSE message followed by an RTSP PLAY message with a speed value > 1 (FF) or < -1 (REW). The video server determines the current position in the normal stream, and then switches to a different source and starts streaming from the current position using the new source.

Note – In FF/REW mode, the sink and processing units remain the same, but the source changes. Stopping a stream through user activity: ■ ■

48

The user can pause a video, move to a different point in the video, and resume play using pause() and resume(). Alternatively, the user can stop the video outright using stop().

Java Stream Assembly Programmer’s Guide • December 2004

Terminating a stream through the activity of a set-top box or human administrator: ■

■

The set-top box can terminate a stream by sending the TEARDOWN request, or an administrator can force a stream to terminate for whatever reason. The video server releases resources allocated to the stream.

Note – Each of the following subsections describes a task that a client handling ondemand video transactions typically must accomplish. For the most part, these tasks are specific to on-demand video transactions. See “Common Development Tasks” on page 27 of this chapter for a discussion of tasks that are common to more than one use case. FIGURE 12 shows the initialization sequence for processing VOD, corresponding to code examples 19 through 23.

Chapter 3

Developing Applications

49

FIGURE 12

50

VOD Initialization Sequence

Java Stream Assembly Programmer’s Guide • December 2004

Initialization The control of the VOD media streams requires at least one processor between the stream source and the stream. The initialization process involves looking up the processor factory in the naming service and using the factory to create an instance of the processor, as shown below. CODE EXAMPLE 19 Initialization, VOD Example // // Get the processor factory from the initial context. // InitialContext root_context = new InitialContext();

JsaComponentFactory processor_factory = (JsaComponentFactory) root_context.lookup("MyProcessorFactory"); // // Create a processor instance from the processor factory. // Attributes processor_attributes = new BasicAttributes(true); processor_attributes.put(new BasicAttribute("Name", "MyIpProcessor")); IPProcessor ip_processor = (IPProcessor) processor_factory.createInstance(processor_attributes);

Note – For more information on using the factory classes to look up components in the naming service, see “Common Development Tasks” on page 27 of this chapter.

RTSP Describe Interaction In the interaction reproduced below, the client queries the server for the details of the content rtsp://wurlitzer.kasenna.com/tahiti.mpg4. The server responds with an SDP description of the content indicating two tracks—201 (video stream) and 101 (audio stream)—along with the details of the tracks. The notation “C-> S” represents the client-to-server interaction, and the notation “S -> C” represents the server-to-client interaction. CODE EXAMPLE 20

RTSP Describe Interaction, Client/Server Communication

C -> S: DESCRIBE rtsp://wurlitzer.kasenna.com/tahiti.mpg4 RTSP/1.0 CSeq: 1 Accept: application/sdp S -> C: RTSP/1.0 200 OK CSeq: 1 Date: 25 Apr 2002 02:27:01 GMT Server: Kasenna MediaBase version 6.0 Content-type: application/sdp

Chapter 3

Developing Applications

51

Content-Base: rtsp://wurlitzer.kasenna.com/tahiti.mpg4/ Content-Length: 1035 v=0 o=NoSpacesAllowed 1 1 IN IP4 10.10.2.44 s=tahiti.mpg4 c=IN IP4 0.0.0.0 t=0 0 a=control:/ a=range:npt=0-378.345000 a=ISMA-compliance:1,1,1 m=video 0 RTP/AVP 97 a=rtpmap:97 MP4V-ES/90000 a=control:trackID=201 a=fmtp:97 profile-level-id=243; config=000001B022000001B509000001000000012000845D4C285820F0A300 a=mpeg4-esid:201 m=audio 0 RTP/AVP 96 a=rtpmap:96 mpeg4-generic/44100/2 a=control:trackID=101 a=fmtp:96 streamtype=5; bitrate=196608; profile-level-id=15; mode=AAC-hbr; config=1210; SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1; a=mpeg4-esid:101

The following code example shows how the server (in this case, a Java Stream Assembly API application) gets the media description information requested by the client. CODE EXAMPLE 21 RTSP Describe Interaction, Server Code, VOD Example // // Create the file source factory // JsaComponentFactory file_source_factory = (JsaComponentFactory) root_context.lookup("MyFileSourceFactory"); // // Create a file source instance from the file source factory. // Attributes file_attributes = new BasicAttributes(true); file_attributes.put(new BasicAttribute("Name", "MyFileSource"));

StreamFileSource file_source = (StreamFileSource) file_source_factory.createInstance(file_attributes); file_source.open("/tahiti.mpg4");

52

Java Stream Assembly Programmer’s Guide • December 2004

In the case of IP transports the data could be stored in a number of formats. Hence, the representation provided by the API is somewhat generic. This is similar to an SDP type description and could in fact be an SDP description. The following example shows how to get media attributes for the stream content. CODE EXAMPLE 22 Get MediaAttributes for Stream Content MediaAttributes media_attributes = file_source.getMediaAttributes();

In the last phase of the RTSP describe interaction, you pass the media attributes object to the processor and get the description that can be sent to the client. It is necessary to involve the processor here because the processor generates the RTP packets and decides the format it can support. CODE EXAMPLE 23 Create the RTP and RTCP Streams Collection transport_description = ip_processor.getTransportDescription(media_attributes);

At this point the application is expected to massage the information in the transport description into a format that the client will expect.

RTSP Setup Interaction The following series of code examples shows how to set up the required source and sink ports for the video stream (track ID 201) identified in the “RTSP Describe Interaction” code example. The steps for setting up track 101 would be similar, but are not shown here for the sake of brevity. Because this example uses RTP, two file sinks are created: one for the RTP stream and one for the control (RTPC) stream. When using transport protocols other than RTP, you create only one file sink. After setting up the required source and sink ports, you must configure the ports at either end of each stream to match, then connect the source and sink ports to the processor. (See the “Moving Data Between Components: Port Types” section of Chapter 2 and “Compatible Port Pairs” on page 33 of this chapter for details on matching port types.).

Setting Up Source and Sink Ports Begin by creating a session object from which to get media streams. FIGURE 12 corresponds to code examples 24 through 27.

Chapter 3

Developing Applications

53

Sequence for Setting Up Source and Sink Ports, VOD Example

54

Java Stream Assembly Programmer’s Guide • December 2004

Note – The getInstance() method is a static method of the SessionManager class. CODE EXAMPLE 24 Creating a session object, VOD Example Session session = SessionManager.getInstance().createSession();

MediaAttributes ma = (MediaAttributes) file_source.getMediaAttributes(); if (!(ma instanceof SDPAttributes)) { throw new IllegalArgumentException( "Unsupported Media Attributes Type"); } SDPAttributes gma = (SDPAttributes) ma; Collection elementaryStreams = gma.getElementaryStreams(); SDPTrackAttributes esa = null; Iterator i = elementaryStreams.iterator();

Next, get the SDP track attributes corresponding to track 201. CODE EXAMPLE 25 Getting the SDP Track Attributes, VOD Example while (i.hasNext()) { SDPTrackAttributes SDPAttr = (SDPTrackAttributes) i.next(); if ("/tahiti.mpg4/trackId=201".equals(SDPAttr.getTrackId())) { esa = SDPAttr; break; } }

if (esa == null) { throw new IllegalArgumentException("Invalid Track"); }

Now, set up the control node on the input side of the processor. CODE EXAMPLE 26 Setting Up the Input Control Node, VOD Example ControlNode processor_input_node = null;

The same control node can be returned for more than one elementary stream, as is the case when the data is multiplexed. For this reason, check next if the control node is already connected. If the node is not already connected, create a stream and a control node and connect them. CODE EXAMPLE 27 Create and Connect Stream and Control Node, VOD Example if (!cn.isConnected()) { strm201 = session.createStream(); processor_input_node = ip_processor.createControlNode(); strm201.connect(cn.getPortFactory(), processor_input_node.getPortFactory()); } else { processor_input_node = cn.getPeer(); }

Chapter 3

Developing Applications

55

Now you are ready to create the net sink factory and net sink instance. FIGURE 13 corresponds to code examples 28 and 29.

FIGURE 13

56

Sequence for Creating NetSink and Opening Ports

Java Stream Assembly Programmer’s Guide • December 2004

CODE EXAMPLE 28 Creating the Net Sink Factory and Instance, VOD Example // Create a net sink factory. JsaComponentFactory net_sink_factory = (JsaComponentFactory) root_context.lookup("MyNetSinkFactory");

// Create a net sink instance from the net sink factory. Attributes net_sink_attributes = new BasicAttributes(true); net_sink_attributes.put(new BasicAttribute("Name", "MyNetSink")); StreamNetSink rtp_sink = (StreamNetSink) net_sink_factory.createInstance( net_sink_attributes); StreamNetSink rtcp_sink = (StreamNetSink) net_sink_factory.createInstance( net_sink_attributes);

Opening and Configuring RTP and RTCP Ports With the source and sink ports set up, you can now complete the setup interaction by doing the following: ■ ■ ■

Open and configure the RTP and RTCP sink ports Create the RTP and RTCP streams Configure the processor to route the streams correctly

CODE EXAMPLE 29 Opening the RTP and RTCP Sink Ports, VOD Example // Open the RTP port rtp_sink.openUDPServer(host, port);

// Open the RTCP port, Note that the port number of the RTCP // port must be one greater than the RTP port port = rtp_sink.getDestinationPort() + 1; rtcp_sink.openUDPServer(host, port); // RTPConfiguration objects are used to distinguish between RTP // control nodes and RTCP control nodes. // RTPConfiguration rtpConfig = new RTPConfiguration(); rtpConfig.setIsRTCP(true); ControlNode rtp_control_node = ip_processor.createControlNode(); ControlNode rtcp_control_node = ip_processor.createControlNode(rtpConfig);

Now you are ready to create the RTP and RTCP streams as shown in FIGURE 14, which corresponds to code examples 30 and 31.

Chapter 3

Developing Applications

57

FIGURE 14

58

Creating Streams and Configuring Router Processing

Java Stream Assembly Programmer’s Guide • December 2004

CODE EXAMPLE 30 Creating the RTP and RTCP Streams, VOD Example // Create the RTP stream Stream rtp_stream = session.createStream();

rtp_stream.connect(rtp_control_node.getPortFactory(), rtp_sink.createControlNode().getPortFactory()); // Create the RTCP stream Stream rtcp_stream = session.createStream(); rtcp_stream.connect(rtcp_control_node.getPortFactory(), rtcp_sink.createControlNode().getPortFactory());

As the final step in the setup interaction, configure the processor to route input media to the designated control nodes. CODE EXAMPLE 31 Configuring Processor Routing, VOD Example // Route RTP stream ip_processor.addRoute(processor_input_node, rtp_control_node, esa); // Route RTCP stream ip_processor.addRoute(processor_input_node, rtcp_control_node, esa);

Playing the Video In the examples presented throughout this section, the subscriber’s video selection (an MPEG4 file) contains two separate media streams: a video stream (track ID 201) and an audio stream (track ID 101). Although these two streams must be opened and connected separately, they must be managed in unison in order to produce correct behavior for the start(), stop(), pause(), resume(), and positioning functions. To accomplish this, the Java Stream Assembly API allows you to define a StreamSet that includes all streams that are part of the same session. CODE EXAMPLE 32

Playing the Video Selection, VOD Example

// // Define the StreamSet // StreamSet strm = new StreamSet(); strm.addStream("201", strm201); strm.addStream("101", strm101); // // This will need to invoke the start() method in the source and the // sink ports. // strm.play();

Chapter 3

Developing Applications

59

Delivering Interactive TV The scenario in this section illustrates a strategy for delivering interactive TV (iTV). In this example, several interactive TV programs are available in a source multi-program transport stream. The interactive TV programs come in two forms: ■

An AV stream and a data stream that have been properly associated with one another by the programming provider.

■

A data stream that forms a programming channel by itself, without an accompanying AV stream.

The task is to multiplex together the AV and data streams to be delivered. FIGURE 15 shows the block diagram for the iTV examples.

FIGURE 15

Block Diagram: iTV

Note – Each of the following subsections describes a task that a client handling iTV transactions typically must accomplish. For the most part, these tasks are specific to iTV transactions. See the “Common Development Tasks” section on page 27 of this chapter for a discussion of tasks that are common to more than one use case. FIGURE 16 shows the initialization sequence for the iTV example, corresponding to code examples 33 and 34.

60

Java Stream Assembly Programmer’s Guide • December 2004

FIGURE 16

Initialization Sequence, iTV Example

Chapter 3

Developing Applications

61

CODE EXAMPLE 33 Initialization, iTV Example // // Get the processor factory from the initial context. // InitialContext rootContext = new InitialContext(); JsaComponentFactory processorFactory = (JsaComponentFactory) rootContext.lookup("MyProcessorFactory"); // // Create a processor instance from the processor factory. // Attributes processor_attributes = new BasicAttributes(true); mpeg2AssemblerAttributes.put(new BasicAttribute("Name", "MyMPEG2Assembler")); MPEG2Assembler myMpeg2Assembler = (MPEG2Assembler) processorFactory.createInstance(mpeg2AssemblerAttributes);

Setting Up Outputs and Inputs FIGURE 17 continues with the I/O setup sequence in code example 34.

62

Java Stream Assembly Programmer’s Guide • December 2004

FIGURE 17

I/O Setup Sequence, iTV Example

Chapter 3

Developing Applications

63

CODE EXAMPLE 34 Setting Up Outputs and Inputs, iTV Example // // Configure the ASI output and set its data rate for the outgoing MPTS // // Lookup ASI Output "LocalDistribution" // JsaComponentFactory asiSnkFactory = (JsaComponentFactory) rootContext.lookup("My Vendor ASI Sink Factory"); // // Create an ASISink instance from the ASI sink factory. // Attributes asiSnkAttributes = new BasicAttributes(true); asiSnkAttributes.put(new BasicAttribute("Name", "Local ASI Distribution")); StreamASISink asiSink = (StreamASISink) asiSnkFactory.createInstance(asiSnkAttributes); // // Lookup ASI Input "LiveFeed" // JsaComponentFactory asiSrcFactory = (JsaComponentFactory) rootContext.lookup("My Vendor ASI Source Factory");

// // Create an ASI source instance from the ASI source factory. // Attributes asiSrcAttributes = new BasicAttributes(true); asiSrcAttributes.put(new BasicAttribute("Name", "Live ASI Feed")); StreamASISource asiSource = (StreamASISource) asiSrcFactory.createInstance(asiSrcAttributes); MediaAttributes liveSrcMediaAttributes = asiSource.getMediaAttributes(); if (!(liveSrcMediaAttributes instanceof MPEG2Attributes)) { throw new IllegalArgumentException("Unsupported Media Attributes Type"); } // // Lookup Carousel StreamSource Factory // // Create the Data Carousel source factory // JsaComponentFactory carouselSrcFactory = (JsaComponentFactory) rootContext.lookup("My Vendor Data Carousel Source Factory"); // // Create a StreamSource by name "Data Carousel Stream Source" // Attributes carouselSrcAttributes = new BasicAttributes(true); carouselSrcAttributes.put(new BasicAttribute("Name", "Data Carousel Stream Source")); StreamSource carouselSource = (StreamSource) carouselSourceFactory.createInstance(carouselSrcAttributes);

64

Java Stream Assembly Programmer’s Guide • December 2004

Session session = SessionManager.getInstance().createSession(); // // Connect ASIInput and Mux // ControlNode asiSrcCn = asiSource.createControlNode(); // // The control node on the input side of the processor // ControlNode m2AssemblerInputNode1 = null; Stream asiSrcMuxStream = session.createStream(); // // Create a control node in the processor and connect // the two components. // m2AssemblerInputNode2 = myMpeg2Assember.createControlNode(); asiSrcMuxStream.connect(asiSrcCn.getPortFactory(), m2AssemblerInputNode2.getPortFactory()); //Connect Carousel Source and Mux // ControlNode crslSrcCn = carouselSource.createControlNode(); // // The control node on the input side of the processor // ControlNode m2AssemblerInputNode1 = null; Stream crslSrcMuxStream = session.createStream(); // // Create a control node in the processor and connect // the two components. // m2AssemblerInputNode1 = myMpeg2Assember.createControlNode(); crslSrcMuxStream.connect(crslSrcCn.getPortFactory(), m2AssemblerInputNode1.getPortFactory()); // // Connect ASIOutput and Mux // ControlNode asiSnkCn = asiSink.createControlNode(); // // The control node on the input side of the processor // ControlNode m2AssemblerOutputNode = null; Stream asiSnkMuxStream = session.createStream(); // // Create a control node in the processor and connect // the two components. // m2AssemblerOutputNode = myMpeg2Assember.createControlNode(); asiSnkMuxStream.connect(asiSnkCn.getPortFactory(), m2AssemblerOutputNode.getPortFactory());

Chapter 3

Developing Applications

65

Configuring Streams, iTV Example In the following code example, we check to make sure the total output bitrate of the multiplexer with the addition of the new carousel source does not exceed the maximum bitrate supported by the ASI output. FIGURE 18 shows the stream configuration and processing sequences for the iTV example, corresponding to code examples 35 and 36.

FIGURE 18

66

Stream Configuration and Processing, iTV Example

Java Stream Assembly Programmer’s Guide • December 2004

CODE EXAMPLE 35 Configuring Streams, iTV Example long carouselSrcRate = 2000000; int dataCarouselPn = 15; long outputBitRate;

// // compute new total output bitrate // InputMediaMPEG2Process asiSrcProcess = MyMpeg2Assembler.getInputMediaProcess(asiSrcCn); outputBitRate = asiSrcProcess.getStreamBitRate(); outputBitRate += carouselSrcRate; // // // // // // // // if

Check to make sure that the output bitrate is within the ASI Output limit, which is returned by asiSink.getTransportBitRate(). If it is over the limit, we reduce the courousel source bitrate so that total output bitrate is within the maximum bitrate supported by the ASI output. (asiSink.getTransportBitRate() < (outputBitRate)) { outputBitRate = asiSink.getTransportBitRate(); carouselSrcRate = outputBitRate asiSrcProcess.getStreamBitRate();

} // Configure the bitrate and program number for the carousel // source process of the multiplexer. // InputMediaMPEG2Process crslSrcProcess = MyMpeg2Assembler.getInputMediaProcess(crslSrcCn); crslSrcProcess.setStreamBitRate(carouselSrcRate); crslSrcProcess.setSPTSProgramNumber(dataCarouselPn); // // Configure the output of the multiplexer to automatically // generate PSI tables, remap PIDs, restamp PCR, set the // PSI table interval, and assign a new transport stream ID. // MyMpeg2Assembler.configureAutoPSI(true); MyMpeg2Assembler.configureRemapPIDs(true); MyMpeg2Assembler.configurePCRRestamp(true); MyMpeg2Assembler.configureForceNIT(asiSrcCn); MyMpeg2Assembler.setPSIInterval(100); MyMpeg2Assembler.setTransportStreamID(200);

Chapter 3

Developing Applications

67

Stream Processing, iTV Example In the following code example, we show how to change the SPTS program number during operation and how to delete a carousel data stream from the multip[lexer output. CODE EXAMPLE 36 Stream Processing, iTV Example // Changing processing in an input // crslSrcProcess.setSPTSProgramNumber(10); // // deleting an input // crslSrcMuxStream.disconnect(); crslSrcMuxStream.finalize(); carouselSource.releaseControlNode(crslSrcCn); myMpeg2Assembler.releaseControlNode(m2AssemblerInputNode1);

68

Java Stream Assembly Programmer’s Guide • December 2004

CHAPTER

4

Developing Components The component developer is responsible for certain tasks. See “Roles and Responsibilities” on page 21 for details. Components are the building blocks of a JavaTM Stream Assembly API application, providing the core media processing elements. There are a number of different types of components, including: ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■

StreamASISink—Defines the ASI sink StreamASISource—Defines the ASI source StreamFileSink—Defines the stream file sink StreamFileSource—Defines the stream file source StreamNetSink—Defines the network sink StreamNetSource—Defines the network source MPEG2Assembler—Defines the MPEG-2 assembler IPProcessor—Defines the IP processor (RTP, etc.) StreamQAMSink—Defines the stream QAM sink IPSink—Defines the IP sink IPSource—Defines the IP source

All components share the same basic design, including these common factors: ■

■

■

■

■

■

Because they derive from JsaComponent, they can be loaded, connected, monitored and controlled in a common way. They correspond to a factory class (JsaComponentFactory), which can be used to create specific instances. They support Ports that act as transport adapters, connecting one component to another. Each JsaComponent must support at least one form of public Port: Local, Directed, or Remote. Their stream control points (ControlNode) must support at least the basic positionless Control interface. At a minimum, this basic control provides a means of starting, stopping, pausing and resuming a stream. They must give control transition callbacks to a provided ControlContext so that multiple controls participating in a larger coordinated control may be synchronized. They must implement authorization logic associated with the Permission classes so an administrator can set up the system authorization model appropriately.

69

Implementation Models and Strategies The first step is to download a copy of the Java Stream Assembly Reference Implementation and the javadocs from http://www.jcp.org/en/jsr/detail?id=158. A JsaComponent implementation is most likely to be split into two distinct pieces: ■

The JsaComponent implementation that the application sees.

■

The processing part, which might reside in another process, on a different host, in a shared library, or in another location.

The means by which these pieces are connected is entirely up to the component provider. The three most likely models are: ■

■

■

70

Use an all-Java™ implementation in-process. In this model, the instance of JsaComponentFactory or JsaComponent can be loaded into LDAP in its entirety. The result of the client lookup is to return the complete implementation. In this situation, the class library must be available in the client application classpath. The component could also implement the Referenceable interface. Delegate command and control to a native-code shared library. This model is similar to the previous one, except that a loadLibrary call is made in a static initializer of the class. The shared library must be accessible through the system load-library path, and the user must be granted permission to load the library. Delegate command and control to an application running on a different process (possibly on a different host). In this model, the JsaComponentFactory and the JsaComponent proxy can be loaded as in the two models described above. Initializing the JsaComponent typically causes a connection to the “real” implementation to be established. In the case of RMI-enabled server implementations, the JsaComponent instance first performs an RMI lookup to obtain a reference to the servant. This second stage lookup should not be confused with the initial factory or stub lookup.

Java Stream Assembly Programmer’s Guide • December 2004

Choosing an Implementation Strategy There are two options for enhancing existing streaming media technology to conform to the Java Stream Assembly: ■

■

Option 1: The Shared Library Model You have an existing piece of hardware (e.g., an ASI board) that you distribute with device drivers, C libraries, and/or library code that is shared with other processes. Option 2: The Separate Application Model You have an existing streaming media application that runs as a separate operating system process. It might have additional command and control GUIs or other supporting applications.

Decide what implementation model you’d like to follow. If you already have C/C++ APIs for your product, you will probably need to use JNI. Create native libraries that call your APIs. It’s logically appropriate to use C++ classes in your implementation, but C libraries will work. The following subsections describe in greater detail the two options introduced above.

The Shared-library Model This option makes sense if your primary development effort is in C or C++. The simplest architectural approach is to wrap the C/C++API with Java using JNI technologies, as described athttp://java.sun.com/j2se/1.5.0/docs/guide/jni/. You can then use the Java layer to implement the required Java Stream Assembly interfaces and delegate through to the C/C++ code using JNI. In addition, consideration must be given to the means by which data is to be moved into or out of the component.There are two common mechanisms available: TCP/IP or ringbuffer. Of the two, the TCP/IP Port is the easiest to implement and should be considered for the first cut of the implementation. In theory the ring-buffer may provide better performance overall. If you use a ring buffer, the C/C++ ring buffer code should be located within the sharedlibrary code developed for this component. The TCP/IP code should also be located within this layer (C/C++), but can be implemented within the Java code for initial exploration.

Chapter 4

Developing Components

71

FIGURE 19 shows a typical shared-library design.

FIGURE 19

72

Shared Library Component Development Model

Java Stream Assembly Programmer’s Guide • December 2004

The primary features of this model are: ■

■ ■

■

The underlying hardware (e.g., the ASI board) must be located on the same host machine as the Java Stream Assembly application. Media streams move through the C/C++ shared library layer. Command, control, and events mapped from Java layer through to underlying C/C++ implementation. Complete assembly (JAR and SO) instantiated within the Java Stream Assembly application.

If a distributed model is more desirable then a model similar to option 2 below is more appropriate.

The Separate Application Model If you identify with option 2 then it is likely that you have products or libraries that process or create streams, e.g., software multiplexer, carousel, file source, etc. The most likely architectural model will be one that extends the existing product to support the stream transport types required for Java Stream Assembly interoperability: e.g.TCP/IP, ring-buffer, etc., and provide some means of accepting command, controls and firing events remotely. For a Java application, this may be an RMI interface; for a C/ C++ application, this could be CORBA or a simple command interpreter activated through a TCP/IP connection. FIGURE 20 shows how an existing software multiplexer might look when enhanced to support Java Stream Assembly.

Chapter 4

Developing Components

73

FIGURE 20

Separate Application Component Development Model

The primary features of this model are: ■

■ ■ ■

The Java Stream Assembly component and its stream processing component can be deployed on different host machines. Media streams moved through the original C/C++ application. Command, control and events mapped from Java application through to C/C++ application through some communication mechanism such as CORBA, RPC, or TCP/IP. Jar file implementation of JsaComponent instantiated within the Java Stream Assembly application, while the stream processing part of the component is hosted with the vendor-specific application.

Modeling The component provider is responsible for implementing lots of classes. Modeling techniques like UML are very valuable as an aid to comprehending the footprint of the Java Stream Assembly architecture. Of particular use are the class diagram shown FIGURE 21 and the sequence diagram shown in FIGURE 22.

74

Java Stream Assembly Programmer’s Guide • December 2004

FIGURE 21

Typical UML Class Diagram

Chapter 4

Developing Components

75

FIGURE 22

76

Typical UML Sequence Diagram

Java Stream Assembly Programmer’s Guide • December 2004

Implementing Ports There are three steps required to implement a port: ■

■

■

Choosing a primary port type—This is based on architectural considerations such as: whether the stream data flows through the Java layer, whether stream data flows over the network, and whether stream data flows between processes using shared memory. Determining if the port is source or sink—This is based on the direction of stream data flow. Determining if the port is push or pull—If the sink port will request stream data from the source port, both ports must be pull. If the sink port will passively consume stream data from the source port, both ports must be push.

These three steps are described in detail in the following subsections.

Choosing a Primary Port Type Ports are divided into four primary types, described in detail in “Choosing a Primary Port Type” on page 77.The key points to consider when selecting a primary port type are speed, interoperability and the topology of the deployment. Directed—Ports communicate with explicit reads and writes from a Java program. ■

Pros: Java layer is able to examine and do processing of the data stream

■

Cons: Throughput may be somewhat limited.

Remote—ports communicate using a network protocols (TCP/IP). ■

Pros: Components may reside on different hosts

■

Cons: Protocol stack overhead and network bandwidth may be an issue.

Local—ports communicate using a ring buffer in POSIX/SYS V shared memory ■

Pros: Fast. Possible to implement zero data copy transfers.

■

Cons: Components must reside on the same physical machine. May require more integration testing with other vendor’s components.

ComponentInternal—ports communicate using an implementer specified mechanism. ■

Pros: Vendor’s native APIs and architecture may be used directly.

■

Cons: Sacrifice interoperability with other vendor’s components.

Determining if the Port is Source or Sink As described in “Streams” on page 9, a port at which stream data originates is a source port, and one at which stream data is consumed is a sink port. For source ports, you must implement the interface SourcePort; for sink ports, you must implement the interface SinkPort.

Chapter 4

Developing Components

77

Determining if the Port is Push or Pull If the sink port will request stream data from the source port, both ports must be pull. If the sink port will passively consume stream data from the source port, both ports must be push. Both ends of a connection are either both push or both pull. The nature of the component probably naturally leads you to choose one or the other. ■

■

A Push Relationship Choose the push model if the source data is time-critical or has no flow control. For example, a StreamASISource receiving from a broadcast feed will always use the push model. The corresponding sink port (which must also use the push model) must consume the data at the rate supplied by the source to avoid overflows and data loss. A Pull Relationship Choose the pull model if the sink port regulates the flow of stream data. A typical example of a pull model is a StreamFileSource connected to a StreamASISink which broadcasts live channels. The source port in such a relationship is assumed to be able to supply data at the required rate or an underflow occurs.

Note – When using the push model, data rates from source and sink must be matched to avoid data loss. You may need to use additional buffering deal with any input delays and jitter.

Using Remote Method Invocation (RMI) The chief benefit of RMI for the Java Stream Assembly component implementer is that RMI permits components to be distributed across different Java virtual machines, on one or more hosts. One useful design pattern is to use RMI to segregate the classes which call native libraries. Place the classes which implement the Java Stream Assembly interface on the client side. Create corresponding classes which call native methods on the server side. This can help to reduce dependencies and unwanted interactions with other vendors’ components running in the same virtual machine. There are a few disadvantages: ■

■ ■

78

You double the number of classes you need to implement—one each for the client and server sides. The RMI mechanism introduces overhead to martial data and make the remote calls. It can be messy if the server side object also has to call methods on the client side (for example to implement call-backs).

Java Stream Assembly Programmer’s Guide • December 2004

Working with Native Objects If you have legacy native implementations or high data rates that require special hardware or low-level processing, you might need to delegate the implementation of certain Java Stream Assembly APIs to native objects. In such cases, the Java objects which implement the component interfaces might end up calling native methods to perform the actions required by the APIs. An important consideration is how to access this native code from within the Java Stream Assembly components and their interfaces. If you have proprietary APIs or legacy native implementations, you will need to provide an adapter from the Java Stream Assembly API implementation to the corresponding native implementation. Your native code may have to persist some sort of contextual information across calls to the API. In order to do this, you will have to keep track of instances and maintain state information. The native API may incorporate asynchronous I/O or signals, which may require call-backs into the Java application. For hardware devices, you will need to do some mapping from the Java Stream Assembly concept of the port to the physical device, typically by making direct calls to the component object itself or by using factory classes.

Naming and Lookup The first task of a client application is to assemble all the components (JsaComponent) or component factories (JsaComponentFactories) so that streams can be established between them. The application looks up components using a naming service. No specific naming service is specified for the Java Stream Assembly API, however. While client access to the naming service should be through JNDI, not all naming services are interchangeable. Because the Java Stream Assembly API does not mandate a specific implementation architecture, no assumptions can be made about its use. LDAP is the most likely (and preferred) open technology for component and component factory storage and lookup.

Providing Security Policies A number of Permission classes are provided as part of the Java Stream Assembly API platform for use by a component provider. For example, the ControlPermission class represents a permission to either control or observe a control. Component providers may elect to enforce these permissions as part of their implementations.

Chapter 4

Developing Components

79

80

Java Stream Assembly Programmer’s Guide • December 2004

CHAPTER

5

JSR-158 Web Resources

Java Stream Assembly API Specification To download the latest version of the API documentation, go to http://www.jcp.org/en/jsr/detail?id=158

Java Stream Assembly Reference Implementation To download the Java Stream Assembly referenceimplementation , go to http://www.jcp.org/en/jsr/detail?id=158

JNDI (Java Naming and Directory Interface) We recommend using JNDI to support the dynamic look-up of JsaComponents and their related factory classes. An LDAP directory implementation is the most likely choice of directory technology. For information on JNDI, go to http://java.sun.com/products/jndi/. For an overview and tutorial on JNDI, go to http://java.sun.com/products/jndi/tutorial/. For information on storing/loading Java objects to/from a directory, see the tutorial at http://java.sun.com/products/jndi/tutorial/objects/index.html. Component providers should consider providing tools to configure and deploy JsaComponents and JsaFactory classes to an LDAP directory.

81

JNI (Java Native Interface) Many existing streaming media products consist of native code libraries. As a result vendors wishing to utilize their functionality within a Java Stream Assembly implementation will need to provide access from Java. Use JNI to map Java to C/C++. For a tutorial, go to http://java.sun.com/docs/books/tutorial/native1.1/index.html.

JMX (Java Management Extensions) While Java Stream Assembly does not currently mandate the use of management tools like SNMP, the use of tools of this type is commonplace in the media industry. The Java platform provides an implementation-independent way to enable applications for management: JMX. For information on JMX, go to http://java.sun.com/products/JavaManagement/index.jsp

RMI (Remote Method Invocation) It is possible for a component implementation to consist of both a client and server tiers.If both these tiers utilize Java technologies, they can be connected using RMI. For more information, go to http://java.sun.com/products/jdk/rmi/index.jsp.

82

Java Stream Assembly Programmer’s Guide • December 2004

Index

A addStream(), 59 administered object, 29 administered objects defined, 29 application defined, 27 Application Assembler, xiv Application Component Provider, xiv applications developing, 27 to 68 applications vs components, 20 ASI, 3 ASI source. See ASI assembly, 4 Asynchronous Serial Interface (ASI), 3 audio stream, 51

B basic stream control, 33 bit-rate out of range events, 37 BitRateOutOfRangeParams, 37 broadcast video, 8 broadcast video applications developing, 38 buffer space, 37

C choosing a port type, 77 client defined, 27 client ceveloper, 24

Client Tier, 20 Clock position type, 18 command interpreter, 73 common application development tasks, 27 to 38 component intermediate, 12 processor, 11 stream sink, 11 stream source, 11 component provider, 22 ComponentPrivate port, 19 components developing, 69 components vs. applications, 20 compression formats, 3 connection details client, 32 connection details provider, 32 connection details user, 32 ConnectionCtrl, 23 ConnectionDetails, 23, 32 Control, 24 Control Factory, 24 control type positionless, 33 control types, 15 pause(), 15, 16 prime(), 15, 17 resume(), 15, 17 start(), 15, 16 stop(), 15, 16 ControlContext, 24, 35 controlling the stream flow, 33 ControlNode, 23, 33 CORBA, 73

83

D data moving, 20 data copies avoiding unnecessary, 4 data-movement model, 11, 20 default strategy, 30 connecting components with, 30 defined, 30 delivery scenarios VOD over IP, 48 Deployer, xiv describe interaction VOD, 51 DESCRIBE RTSP request, 48 developer roles, xiv directed port, 10, 19, 30

E elementary stream, 1 Endpoint, 22 EventListeners, 24 events bit-rate out of range, 37 monitoring, 35 overflow, 37 underflow, 37

F FF stream, 48 FF/REW mode, 48 switching to, 48 file source, 12 file source instances, 29 fileStreamComp, 29 FileStreamFactory lookup, 29

G getPosition(), 17 getScale(), 17 Gigabit Ethernet, 3

J Java Management Extensions, 82 Java Naming and Directory Interface, 81 Java Native Interface, 82 Java platform-enabled (directed) port, 19 Java Stream Assembly API design requirements, 8 limit of scope, 9 purpose, 7 JMX, 82 JNDI, 81 JNI, 82 JSAComponent, 22 JsaComponent, 29 lookup, 28, 29 JSAComponentFactory, 22 JsaComponentFactory lookup, 29 JSR 158, xiii, xiv

L local port, 10, 19, 30 logical node, 11 lookup, 29, 79 FileStreamFactory, 29 JsaComponent, 28, 29 JsaComponentFactory, 29 lookup services, 28

M

I Immediate position type, 18 initialization

84

VOD, 51 input port, 9, 11 Integration Tier, 20 Interactive Television, 8 intermediate component, 12 IP, 4 delivering media over, 2 delivery scenarios, 48 IP network, 2 IPProcessor, 69 IPSink, 69 IPSource, 69 iTV. See Interactive Television.

matching port types, 20 matching streams, 53

Java Stream Assembly API Programmer’s Guide • December 2004

media delivery mechanisms, 4 media streams, 1 definition, 1 modifying data within, 5 MPEG-2 TS, 47 MPEG-4, 47 starting, 5 stopping, 5 modeling the classes, 74 moving data, 20 MPEG, 3 MPEG-2, 3, 4, 48 Elementary Stream, 9 MPEG2Assembler, 69 MPEG-4, 3, 47, 48 MPTS, 2, 9 multiplex, 1 multi-program, 1 single program, 1 multiplexers controlling, 5 feeding data to, 5 monitoring performance, 5 multi-program multiplex, 1 multi-program transport stream, 2, 9

N naming, 79 native objects, 79

O object factory, 29 objects Session, 28 on-demand video applications developing, 47 to 59 on-demand video, defined, 8 output port, 9, 11 overflow events, 37

P package javax.mediastream.component, 25 javax.mediastream.control, 25 javax.mediastream.control.factory, 25 javax.mediastream.control.position, 25 javax.mediastream.endpoint, 25

javax.mediastream.events, 25 javax.mediastream.format, 25 javax.mediastream.port, 25 javax.mediastream.processor, 25 javax.mediastream.stream, 25 javax.mediastream.stream.events, 25 Packet ID, 9 Packet position type, 18 pass-through, 13 PAT, 9 pause(), 15, 16, 48 physical inputs, 29 PID, 9 platform provider, 21 PLAY RTSP request, 48 play(), 59 playing the video VOD, 59 PMT, 9 Port, 23 port as logical node, 11 directed model, 10, 19 input, 9 local model, 10, 19 output, 9 pull sink, 37 pull source, 32 push sink, 32 push source, 32, 37 remote model, 10, 19 sink, 9, 10, 20 source, 9, 10 port type ComponentPrivate, 19 sink, 32 source, 32 port types choosing primary, 77 matching, 20 PortFactory, 23 ports configuring, 53 PortType, 32 Position class, 18 position type, 18 position types, 18 clock, 18

Index

85

immediate, 18 packet, 18 Position, 18 Sample, 18 position-based stream control, 34 primary type, 32 prime(), 15, 17 processing, 3 processor, 11, 12 Product Provider, xiv Program Association Table, 9 Program Map Table, 9 program stream, 1 provisioning VOD, 48 pull sink, 37 pull sink port, 32 pull source port, 32 push sink port, 32 push source, 37 push source port, 32

R Remote Method Invocation, 82 remote port, 10, 19, 30 Resource Tier, 20 restamping, 13 resume(), 15, 17, 48 REW stream, 48 RMI, 73, 78, 82 roles, xiv client ceveloper, 24 component provider, 22 platform provider, 21 RTP, 48 channel, 2 RTP/UDP, 4, 48 RTSP describe interaction (VOD), 51 PAUSE message, 48 request, 48 DESCRIBE, 48 PLAY, 48 SETUP, 48 session, 2 setup interaction (VOD), 53 transaction, 2

86

S Sample, 18 Sample position type, 18 SDP description, 51 security policies providing, 79 separate application model, 73 Session, 21 Session object, 28 SessionManager, 21, 28 setup interaction VOD, 53 SETUP RTSP request, 48 shared-library model, 71 shared-memory (local) port, 19 single program multiplex, 1 single program transport stream, 2, 9 sink port type, 32 stream, 12 sink port, 9, 10, 20 connecting, 53 pull, 32 push, 32 source, 32 file, 12 port type, 32 stream, 12 source port, 9, 10 connecting, 53 pull, 32 push, 32 specific port type, 32 speed value, 48 SPTS, 2, 9 start(), 15, 16 starting a stream VOD, 48 start-up VOD, 48 state machine, 16 stop(), 15, 16, 48 stopping a stream set-top box or administrator, 49 through user activity, 48 Stream, 21 stream establishing between components, 29

Java Stream Assembly API Programmer’s Guide • December 2004

merging, 13 stream control, 15 basic, 33 position-based, 34 types, 15 stream flow controlling, 33 stream flow events monitoring, 35 stream sink, 11, 12 stream source, 11, 12 StreamASISink, 69 StreamASISource, 69 StreamFileSink, 69 StreamFileSource, 69 StreamNetSink, 69 StreamNetSource, 69 StreamProcessor, 22 StreamQAMSink, 69 streams, 9 StreamSet, 21 StreamSet(), 59 StreamSink, 22 StreamSource, 22

Video On Demand (VOD). See on-demand video video stream, 51, 53 VOD applications, developing. See on-demand video applications. VOD. See on-demand video.

T TCP, 48 TCP/IP (remote) port, 19 TEARDOWN request, 49 transport mechanisms, 4 Transport Stream, 9 transport stream, 1 TS, 9

U UDP, 48 UML, 74 to 76 underflow events, 37 user-defined strategy connecting components with, 32

V VCR control, 15 vendor-specific (ComponentPrivate) port, 19 video playing, 59

Index

87

88

Java Stream Assembly API Programmer’s Guide • December 2004