SESSION 1: MIDDLEWARE HISTORY AND TECHNICAL OVERVIEW

Michael Schonberg OpenMAMA Maintainer

April 4, 2012

NYSE Technologies

Agenda Session 1 (09:00-10:00)  Introduction  A Brief History of Message Oriented Middleware  Messaging Concepts  OpenMAMA: Highlights  OpenMAMA: Architecture  OpenMAMA: Key Constructs Overview Session 2 (10:00-11:00)  OpenMAMA: Key Constructs Details  OpenMAMA: Writing Code

Introduction

Introduction OpenMAMA is:  Middleware Agnostic Messaging API  Supports a variety of Message Oriented Middleware (MOM) platforms    

  

MOM allows applications to exchange information without establishing explicit connections

De-facto standard in financial services A consistent abstraction layer High performance Open Source Hosted by the Linux Foundation Supported by a diverse group of industry stakeholders

History Lesson Presented by Feargal O’Sullivan

The Idea

The hardware BUS architecture… applied to software

Two Main Lines Publish/Subscribe

Queues

Made Famous By: Teknekron Software Systems (nka TIBCO Software)

Made Famous By: International Business Machines (aka IBM)

• • • • • • •

Client-Server Daemon Based Broadcast/Multicast Event Driven High-speed Topic Naming Scheme ‘Reliable’ Delivery

• • • • • • •

Mainframe to Client-Server Hub & Spoke Point-to-point Guaranteed Delivery Slow (but steady) Queue Names Queue over-flow

Common Features • • • • •

Decoupling Fault tolerance Fire and forget Payload encoding Entitlement control

Hybrids      

Certified Delivery on Pub/Sub Hub & Spoke on Pub/Sub Content Routing Daemonless Pub/Sub Replay Capabilities Last Known Value Cache – Distributed Caches

Multiple Offerings          

TIBCO Rendezvous IBM MQ Series (WebSphereMQ) JMS: TIBCO EMS; Sonic MQ etc. 29West LBM (now Informatica) AMQP: Red Hat MRG ZeroMQ – SonicMQ Solace Systems (FPGA Acceleration) Real-time Innovations (RTI) NYSE Technologies Data Fabric Etc.

Applied Middleware

Quote

10

Custom FX Rates

Feed Handler

SuperFeed

Conflation Cache

Cache

Desktop Traders

Algo

Market Gateway

Market Gateway

Historical Tick Capture

Algo

Market

Trade Gateway

Algo

M o n i t o r i n g

Messaging Concepts Presented by: Mike Schonberg

Message Oriented Middleware 

Publish-subscribe semantics  Publishers send messages • Broadcast semantics

 Subscribers receive messages • Topic vs. content based • OpenMAMA supports topic based 

Request-reply  Publishers broadcast  Subscribers optionally send reply directly to publisher



Location independent

Publish-Subscribe



Topic resolution is automatic  Publishers require no awareness of subscribers  Subscribers have no explicit knowledge of publishers

Request-Reply

 

Request is a normal message with “return address” Subscribers optionally reply directly to requester • Point-to-point

Transports and Implementations 

Transports     



TCP UDP Multicast RDMA Shared Memory/IPC IB-Verbs

Implementations  NYSE Technologies Data Fabric • TCP, RDMA, Shared Memory, IB-Verbs

 TIBCO Rendezvous and FTL  AMQP  IBM Websphere MQ

OpenMAMA: Highlights

What is OpenMAMA? 

Open Source Middleware Agnostic Messaging API  Hosted by Linux Foundation  LGPL 2.1



Single API for event-driven messaging  Multiple middlewares and payloads  High performance

  

Mature Widely deployed in financial services Potential applications outside finance

Why Use OpenMAMA?  

Eliminate vendor-lock in Flexibility  Write once for multiple middlewares • Develop to a single open API • Deploy over multiple middlewares

 Simplify IPC 

Focus on critical business logic

More than an just an API 

Advanced data quality support  Gap detection  State recovery

  

Dictionary support Initial images Advanced feature implementation    

Queues and dispatchers Timers IO events Etc.

Applications 

Financial Services  Market data distribution  Order routing  Risk management



Other  HPC/parallel data processing  Logistics  Any application requiring data distribution to heterogeneous consumers

OpenMAMA: ARCHITECTURE

OpenMAMA Architecture

OpenMAMA: Key Constructs

Key Constructs    

Transports Queues Publishers Events    

Subscriptions Inbox (for request/reply) Timers IO events

Q&A [email protected]

SESSION 2 – UNDER THE HOOD Mike Schonberg

Agenda       

Transports Queues Publishers Events Value Added Components Code Introduction to Bridges

Transports 

Establish connections connections between publishers and subscribers  Mechanism varies widely between middlewares • NSD (NYSE Technologies Data Fabric) • Broker based (Avis, AMQP, TIBCO Rendezvous) • Multicast topic resolution (Informatica LBM)



Global configuration and tuning  Buffer sizes  Network parameters  Throttle rates

Queues 

Queue based threading model  One dispatch thread per queue  Several dispatch modes • Single event: invoke one callback and return • Timed: dispatch until empty or timeout • Continuous: dispatch forever

 

Callback based Advanced features  Watermarks to monitor queue depth  Enqueue callback to integrate with other event loops



Custom callbacks

Publishers      

Send messages and requests One publisher per topic Associated with a specific transport Thinnest possible wrapper to native publishing Support throttled publishing Advanced publishing framework  dqpublisher in source  Stateful (market data publishing)

Subscriptions 

Subscription to a single topic  Receive message from all publishers of topic

   

onMsg() callback invoked for each message Creation throttled Optional wildcard support Basic vs. stateful (market data)  Stateful receive initial state  Stateful monitor sequence numbers

Inbox 

Receives replies through onMsg() callback  Multiple replies possible • From different subscribers • From the same subscriber

 

Parameter to publisher_sendFromInbox() Supports multiple requests

Timers  

Repeating timers Interval is floating point in seconds  Resolution dependent on native middleware or OS



onTimerCallback()  Enqueued on most once per interval  Queue depth makes non-deterministic



OpenMAMA provides default implementation

IO Events 

Asynchronous IO support  For non-blocking file descriptors and sockets

  

Invokes callbacks for read and write readiness Support is optional Integrates asynchronous IO into OpenMAMA event dispatching

OpenMAMA: Value-added Components

Other Components 

  

Stateful market data subscriptions/data quality Advanced publishing framework Data dictionary Fault tolerance

Stateful Subscription



Initial value delivered point-topoint



Complex sequence number logic



Other potential applications

Data Quality  

Essential for Stateful/Market Data Subscriptions Sequence number monitoring  Request recap on gap detection  Recap is an initial image

  

Pre-initial spooling Service disruptions from native middleware Callbacks inform application of data quality events  STALE -> data definitely lost  POSSIBLY_STALE -> data may be lost

Advanced Publishing 

   

Framework for publishing to stateful/market data subscribers Sequence number and status fields for data quality Initial value requests Recap requests Refresh messages  Opitional

Data Dictionary 

OpenMAMA supports self describing messages  Field type, name, id (integer) and value

 

Sending names with every msg -> big messages Dictionary maps id’s to names, types and values  From a server as a message  Loaded from a file  Built dynamically



Allows integration of non-self describing payloads

Fault Tolerance   

Applications can run in fault tolerant groups Every instance has a unique weight Heart beat messages determine failure  UDP multicast  Bridge using OpenMAMA and native middleware

 

Callbacks inform instances of transitions Application responsible for recovery logic

OpenMAMA in Action

Let’s write a program! 

Create a publisher  Open a session  Pack a payload  Send message



Create a subscriber  Open interest in a topic  Receive message callback  Unpack message

OpenMAMA Publisher Example 









Taken from mamapublisherc.c example Publishes a message at regular interval Subscribes to a request topic Sends reply on receiving a request Key concepts     

Publishing Subscribing Replying to requests Creating messages Timers

int main(void) { initializeMama (); createIntervalTimer (); createInboundSubscription (); createPublisher (); mama_start(gMamaBridge); return 0; }

Initialization 

Load middleware bridge

void initializeMama(void) { mama_status status; status = mama_loadBridge(&gMamaBridge, gMiddleware);

 By name “wmw”, “lbm”, “tibrv” or “avis”  



if (status != MAMA_STATUS_OK) { printf ("Error loading bridge: %s\n", mamaStatus_stringForStatus (status)); exit (status); }

Initialize OpenMAMA Create or get queue Create a transport

status = mama_open(); mama_getDefaultEventQueue(gMamaBridge, &gMamaDefaultQueue); status = mamaTransport_allocate(&gTransport); status = mamaTransport_create(gTransport, gTransportName, gMamaBridge);

• Two step 1.

2.

Allocate Create (initialize)

}

Create a Timer 

Initialize timer  Default queue from initialization  1 second interval  Repeating



Callback publishes message

static void createIntervalTimer(void ){ status = mamaTimer_create(&gTimer, gMamaDefaultQueue, timerCb, 1.0, NULL); } static void MAMACALLTYPE timerCb(mamaTimer timer, void* closure){ publishMessage (NULL); }

Create a Subscription 

Subscription for requests

static void createInboundSubscription (void){ gInboundCb.onCreate gInboundCb.onError gInboundCb.onMsg gInboundCb.onQuality gInboundCb.onGap gInboundCb.onRecapRequest

 Send reply for each request 

 

Two step create msgCb for each message

createCb ; errorCb; msgCb; NULL; NULL; NULL;

mamaSubscription_allocate (&gSubscription ); mamaSubscription_createBasic (gSubscription , gTransport , gMamaDefaultQueue , gInboundCb , “REQUEST_TOPIC”, NULL);

Callbacks  No messages before createCb

= = = = = =

} static void MAMACALLTYPE inboundMsgCb(mamaSubscription subscription, mamaMsg msg, void* closure, void* itemClosure ){ if (!mamaMsg_isFromInbox (msg)) { return; } publishMessage (msg); }

Publishing 

Publisher requires transport and topic. 



Optional source and root

Sending Messages 

Static message reused



Standard MAMA fields: Status and SeqNo

static void createPublisher(void ){ mamaPublisher_create (&gPub , gTransport , “MAMA_TOPIC”, NULL, NULL); } static void publishMessage(mamaMsg request){ static int msgNumber = 0; static mamaMsg msg = NULL; if (msg == NULL) { mamaMsg_create (&msg); } else { mamaMsg_clear (msg) ; } mamaMsg_addI32 (msg, MamaFieldMsgStatus.mName , MamaFieldMsgStatus.mFid , MAMA_MSG_STATUS_OK);



Custom field: Publisher Topic

mamaMsg_addI32 (msg, MamaFieldSeqNum.mName , MamaFieldSeqNum.mFid , msgNumber++); mamaMsg_addString (msg, "PubTopic", 10002, “MAMA_TOPIC”);



request != NULL: called from msgCb(). Point-to-point reply



Request == NULL: called from timerCb(). Multicast message.

if (request){ mamaPublisher_sendReplyToInbox (gPublisher , request, msg); } else { mamaPublisher_send (gPublisher , msg) ; } }

What Does it Do?



Timer callback fired every second 



Sends a message on “MAMA_TOPIC”

Listens for messages on “REQUEST_TOPIC” 

Message callback fired •

If inbound message is a request then send reply

Middleware Bridges - Overview

Bridges 

Payload and Message Bridges  Directly reflect top-level API • Messages, subscriptions, queues, timers, etc.

 One-to-one function mapping 

Virtual function table  Dynamic loading and static linking  Multiple bridges simultaneously  OpenMAMA macros build/load table • Bridge implementations just provide methods

Questions?

[email protected]