Power Programming with RPC

John Bloomer Technische Unlversitat Darmstadt FACHBEREICH INFORMATIK

B I B L I O T H E K Inventar-Nr.: Sachgebiete:. Standort:

O'Reilly & Associates, Inc. Cambridge

• Koln

Paris

• Sebastopol • Tokyo

Table of Contents Preface Scope of This Handbook Some Assumptions About the Reader Software Mentioned in This Book Examples Additional Sources of Information Conventions Acknowledgments

Chapter 1: Introduction to Remote Procedure Calling Client/Server Computing Remote versus Local Procedure Calling RPC Application Development Defining the Protocol Developing Server and Client Application Code Compiling and Running the Application RPC Makes Interprocess Communications Less Painful How RPC Systems Work RPC Systems and the OSI Reference Model What is "State" and Why is it Important? Top Twenty Distributed Computing Terms and Acronyms

Chapter 2: Network Computing Today Distributed Computing Standards Features of a Distributed Computing Environment NCS :. ; ONC Examining ONC and NCS Machine-independent Data Representation RPC Mechanism Protocol Compilers: The Most Important Feature Authentication Services Network Resource Naming Services Network Time Service

xxv xxvi xxx xxx xxxii xxxiii xxxiv xxxv

1 1 2 3 4 6 9 10 11 13 15 16

19 19 20 21 22 24 24 26 29 30 30 30

Distributed File System

31

There May be a Common Application Environment in Your Future

31

Netwise: A Second ONC Alternative Machine-independent Data Representation RPC Mechanism Protocol Compiler Authentication Services Network Resource Naming Services Network Time Service Distributed File System Summary of Distributed Computing Environments

32 33 33 34 34 34 35 35 35

Rapid Evolution

38

Which One Should You Choose?

40

Put the ONC Suite on Your Machine for Free

Chapter 3: Developing High-level RPC Applications Development Overview From Local to Remote Directory Reading Define the Protocol Data Types Program, Procedure, and Version Numbers High-level ONC RPC Library Calls At the Server At the Client Shared XDR Routines Using the XDR Library Writing the Client and Server Programs At the Server At the Client Compile, Link, and Run Why High-level Calls and Not Protocol Compiling? Some Limitations of the High-level Calls TCP Transport Requires Lower-level Calls

40

43 43 44 -.47 47 47 48 49 50 50 51 54 54 55 56 58 58 59

Chapter 4: Protocol Compiling and Lower-level RPC Programming

61

Development Overview

63

Filename Conventions and Make Using RPCGEN

63 65

The Protocol Definition Language Definitions Symbolic Constants Enumerations Structures

68 68 69 69 70

Unions

71

Typedefs

71

Programs

72

Declarations Special Cases Booleans Strings

74 76 76 77

Opaque Data

.-

Voids

77 78

Preprocessor Symbols and Control

78

Lower-level ONC RPC Library Calls At the Client At the Server An Example: One Client Talks to One Server Debugging

79 79 80 81 89

'....

Step 1: Debug Without the Network

90

Step 2: Use the Raw Transport Step 3: Debug Over the Network

94 97

Deploying Servers During Development

100

Real RPC Power Means Using IPC

101

Chapter 5: UNIX Networking and Interprocess Communication A Network Protocol Primer Internet Addressing Connectionless at the Lowest Level

103 104 104 105

Internet Protocols

105

Applications Protocols

107

Connections and Well-known Ports Types of Servers Network Transport Selection: UDP or TCP?

107 108 109

Adding a Server to the System inetd and Other Lurking Network Daemons Installing a Server Yourself A Digression on Remote Execution Daemons

110 Ill 113 113

Configuring inetd An Overview of UNIX Interprocess Communication

114 119

Pipes and FIFOs Message Queues, Semaphores, and Shared Memory

119 120

Berkeley Sockets and System VTLI Data Representation or Byte Ordering Retrieving Host, Network, and Address Information Getting ONC RPC Information

121 121 122 124

Berkeley Sockets

:..

.•>'

Yeah, But How Do I.Use Sockets? Socket Examples Advanced Socket Programming Issues Remote Execution, Security, and Authentication

Chapter 6: Application Development: Networked Parallel Image Processing

125 127 128 142 144

147

Developing Parallel Algorithms for a Multi -server Network 148 A Simple Model For Parallel Processing on a Network 149 System Requirements and Network Constraints 151 Server Access 151 Server and Network Performance 151 Brute-force Scheduling Using Process Control 152 Programming Asynchronous and Concurrent Processing at the Client .... 152 Making Use of Timers to Watch Child Processes 156 Development Steps 160 Remote Image Processing (RIP) 164 Specifying Filter Coefficients 164

rip Development: It's as Easy as 1 2 3 Step 1: Define the Protocol Step 2: Building the Client Procedure Step 3: Developing the Server Procedure

166 166 168 179

Testing and Running the Program Extending RIP

183 184

Fast SunView Frame Buffer Access Needed

185

Chapter 7: Distributing Existing Applications A Local Image Manager The Header File Modularity Functions Compile, Link, and Run Moving a Local Application to the Network Answer Fourteen Questions First The Strategy

Chapter 8: Managing RPC Servers How to Start the Remote Server Shell Scripts For Starting Servers Starting a Remote Server From within Your Client Application Terminating Your Services System Error: "%STF-E-OPENIN, Server Too Fat" Hit Reset Report Server Information with rpcinfo Changes Under TIRPC Data Sharing: NFS versus Sending it Yourself Host-qualified Filenames

Chapter 9: Multiple Clients and Servers Remote Asynchronous Calls, Multi-server Processing ONC RPC Support The Follow-up RPC Multi-tasking at the Server Multi-tasking with Child Processes Alternatives to Avoid Run-time Process Creation

187 187 188 189 192 197 197 197 200

215 216 217 221 223 225 226 227 231 231 232

235 235 236 250 262 263 267

IX

Combining Asynchronous and Multi-tasking at the Server

268

Lightweight Processing Remote Asynchronous Calls with LWP

270 271

A Minimal Set of LWP Routines

271

Client Multi-server Example

273

Server Multi-tasking Example Caveats

278 282

Chapter 10: RPC Under Windowing Systems The X Window System , X Toolkit Client Application Flow of Control Low-level RPC and X Protocol Similarities Strategies for Using RPC Under X Placing and Servicing RPCs in an Event-driven Environment RPCandXView Synchronous RPCs with a Timer Remote Asynchronous Calls, FRPC Polled with a Timer The Event Notifier and Associated Complications Remote Asynchronous Call Servers Using notify_enable_rpc_svc() RPCandXol/Xt Watching IPC with XtAppAddInput(3Xt) Remote Asynchronous Calls, Servers Using XtAppAddlnputO Comparing Network Windowing Systems and RPC A Digression: Performance of Typical versus Network Windowing Systems Windowing System Evolution Can Hide a Frame Buffer Xll Pixmaps versus SunView Pixrects Augment Typical Windowing Systems with RPCs

Chapter 11: ONC Transport-independent RPC Maintains the ONC RPC Protocol Run-time Transport Independence Network Selection Uniform Addressing TIRPCAPI Backward Compatibility Levels of the Library

283 284 284 285 285 286 290 290 292 296 298 300 301 308 313 314 314 315 322

323 324 325 326 326 326 327 327

An Example Availability

330 332

Chapter 12: Advanced Programming Issues Authentication and How to Use It ONC RPC Credentials and Verification at the Client Adding Authentication to the dim Client Authentication at the Server Adding Access Control to the dim Server Error Reporting Summarized Fault Tolerance, Connection Errors, and Crash Recovery Connection Errors and Recovery Caching Replies at the Server Broken Connections and Testing Crash Recovery

The ONC RPC Programming Reference

333 333 334 339 344 346 350 351 351 353 353 356

359

Section One: ONC XDR Library Routines

36l

Section Two: ONC Portmap Library Routines

377

Section Three: ONC RPC Library Routines

381

Section 1: ONC XDR Library Routines

361

Overview XDR Streams and Their Management Conversion Filters '. Synopsis : xdr_array() xdr_bool() xdr_bytes() xdr_char() xdr_destroy( ) xdr_double() xdr_enum() xdr_float() : xdr_free() xdr_getpos() xdr_inline()

:

36l 363 364 368 368 368 368 369 369 369 369 369 369 370 370

XI

xdr_int() xdr_long() xdrmem_create() xdr_opaque() xdr_pointer() xdrrec_create() xdrrec_endofrecord( ) xdrrec_eof() xdrrec_readbytes() xdrrec_skiprecord() xdr_reference( ) xdr_setpos() xdr_short() xdrstdio_create( ) xdr_string() xdr_u_char() xdr_u_int() xdr_u_long() xdr_union() x.dr_u_short() ••• xdr_vector() xdr_yoid() xdr_wrapstring()

:

Section 2: ONC Portmap Library Routines Overview Synopsis pmap_getmaps( ) pmap_getport( ) pmap_rmtcall() pmap_set() pmap_unset() xdr_pmap() xdr_pmaplist()

xn

;

370 370 370 371 371 371 371 372 372 372 372 372 373 373 373 373 373 374 374 374 374 374 375

3 77 377 378 378 378 378 378 379 379 379

Section 3: ONC RPC Library Routines Overview Functional Summary Building Client Authentication Making the Call from the Client CLIENT Handle Management Server Registration With The Portmap SVCXPRT Service Transport Handle Management Server Side Error Handling And Reporting Server I/O and Utility Direct XDR Access Making Secure RPCs Synopsis authdes_create() authdes_getucred() auth_destroy() authnone_create() authunix_create() authunix_create_default() callrpcO clnt_broadcast() clnt_call() clnt_control() clnt_create() clnt_create_vers() clnt_destroy() ....: clnt_freeresO clnt_geterr() clnt_pcreateerror() clnt_perrno() clnt_perror() clntraw_create() clnt_spcreateerror() clnt_sperrno() clnt_sperror() clnttcp_create()

381

."

381 381 381 382 383 383 384 384 385 386 386 387 387 388 388 388 388 388 389 389 389 390 391 391 391 391 392 392 392 392 392 393 393 393 393

clntudp_bufcreate() clntudp_create() get_myaddress() getnetnameO getrpcportO host2netname() key_decryptsession() key_encryptsession() key_gendes() key_setsecret() netname2host() netname2user() registerrpcO rpc_createerr svc_destroy() svcfd_create() svc_fds svc_fdset svc_freeargs() svc_getargs() svc_getcaller() svc_getreq() svc_getreqset() svcerr_auth() svcerr_decode() svcerr_noproc() svcerr_noprog() svcerr_progvers() svcerr_systemerr() svcerr_weakauth() svcraw_create() svc_register() svc_run() svc_sendreply() svctcp_create() svcudp_create()

:

393 394 394 394 394 395 395 395 396 396 396 396 396 397 397 397 397 397 398 398 398 398 398 399 399 399 399 399 400 400 400 400 400 401 401 401

svcudp_bufcreate() svc_unregister()

401 402

user2netname()

402

xdr_accepted_reply() xdr_authunix_parms()

402 402

xdr_callhdr() xdr_callmsg() xdr_opaque_auth() xdr_rejected_reply() xdr_replymsg() xprt_register()

•.

xprt_unregister() Error Codes

402 402 403 403 403 403 403 404

Appendix A: Obtaining RFCs (Internet Standards, Request for Comment) 407 Appendix B: An RPC Case Study: Networked Ray Tracing

409

Introduction to Ray Tracing 409 Accelerating Ray Tracing 411 Multiple Processor Ray Tracing and Data Distribution 413 Networked Ray Tracing Using RPC : 415 Dynamic Scheduling and Load Balancing 417 But Why Scan-line Parallelism and Not Frame Parallelism? 418 Performance Results •. 421 Conclusions : 421 References 423 Documentation and Source Code 426 Comments on Augmenting the Client to Run Under XI1 with the XView Toolkit 451

Appendix C: Generalized Server Initialization, Inquiry, and Removal Appendix D: Parallel Processing In A Nutshell

453 457

Parallelism

458

Interprocess Control

459

Interprocess Communication

459 xv