HPE NonStop cF SSL Library Reference Manual

Part Number: 839769-002 Published: May 2016 Edition: 1.7.1

© Copyright 2015, 2016 by Hewlett Packard Enterprise Development LP The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprise website. Acknowledgments Itanium® is a registered trademark of Intel Corporation in the United States and other countries. UNIX® is a registered trademark of The Open Group.

Contents Preface

5

What is NonStop cF SSL Library ..................................................................................................................... 5 Understanding the Use of SSL and TLS ........................................................................................................... 5 Who Should Read This Guide .......................................................................................................................... 5 Related Reading ................................................................................................................................................ 6 Document History ............................................................................................................................................. 6

Using NonStop cF SSL Library

7

Overview........................................................................................................................................................... 7 Example Socket code prior to using the NonStop cF SSL Library ......................................................... 7 Client Example Code using NonStop cF SSL Library............................................................................ 8 Server Example Code using NonStop cF SSL Library ........................................................................... 8 Pass-Through Mode ................................................................................................................................ 9 Step by step guide for adding of NonStop cF SSL Library............................................................................... 9 Install NonStop cF SSL Library files...................................................................................................... 9 Include library header file ..................................................................................................................... 10 Initialize the library once ...................................................................................................................... 10 Replace socket and nowait calls with calls to the library ...................................................................... 10 Activate SSL for a given socket............................................................................................................ 11 Error handling ....................................................................................................................................... 11 Bind in the library ................................................................................................................................. 11 Configuration of NonStop cF SSL Library ..................................................................................................... 12 Pass-Through Mode .............................................................................................................................. 12 NonStop cF SSL Library Parameter Reference .................................................................................... 12 Licensing .............................................................................................................................................. 13

Function Reference

14

Replacement of existing calls ......................................................................................................................... 14 SSL_accept1 ......................................................................................................................................... 14 SSL_accept_nw .................................................................................................................................... 14 SSL_accept_nw1 .................................................................................................................................. 14 SSL_accept_nw2 .................................................................................................................................. 14 SSL_bind .............................................................................................................................................. 14 SSL_bind_nw ....................................................................................................................................... 14 SSL_connect_nw .................................................................................................................................. 14 SSL_connect1 ....................................................................................................................................... 15 SSL_connect_nw .................................................................................................................................. 15 SSL_getpeername ................................................................................................................................. 15 SSL_getpeername_nw .......................................................................................................................... 15 SSL_getsockname................................................................................................................................. 15 SSL_getsockname_nw .......................................................................................................................... 15 SSL_getsockopt .................................................................................................................................... 15 HPE NonStop cF SSL Library Reference Manual

3

Contents

SSL_getsockopt_nw ............................................................................................................................. 15 SSL_recv .............................................................................................................................................. 15 SSL_recv_nw ........................................................................................................................................ 15 SSL_send .............................................................................................................................................. 15 SSL_send_nw ....................................................................................................................................... 15 SSL_send_nw2 ..................................................................................................................................... 15 SSL_setsockopt..................................................................................................................................... 16 SSL_setsockopt_nw .............................................................................................................................. 16 SSL_shutdown1 .................................................................................................................................... 16 SSL_shutdown_nw ............................................................................................................................... 16 SSL_socket ........................................................................................................................................... 16 SSL_socket_nw .................................................................................................................................... 16 SSL_socket_get_len.............................................................................................................................. 16 SSL_socket_ioctl .................................................................................................................................. 16 SSL_socket_ioctl_nw ........................................................................................................................... 16 SSL_AWAITIOX ................................................................................................................................. 16 SSL_CANCELREQ.............................................................................................................................. 16 SSL_FILE_CLOSE_ ............................................................................................................................ 16 SSL_FILE_COMPLETE_ .................................................................................................................... 16 SSL_SETMODE................................................................................................................................... 17 SSL_SETMODENOWAIT .................................................................................................................. 17 Calls added by NonStop cF SSL Library ........................................................................................................ 17 SSL_init ................................................................................................................................................ 17 SSL_start .............................................................................................................................................. 18 SSL_SetParam ...................................................................................................................................... 18 SSL_get_encryption_method................................................................................................................ 19 SSL_get_last_error_code ...................................................................................................................... 19 SSL_get_last_error_text ....................................................................................................................... 19

Error numbers and Error texts within NonStop cF SSL Library

21

Introduction ..................................................................................................................................................... 21 Errors running as SSL client ........................................................................................................................... 21 Verification of server certificate failed ................................................................................................. 21 No shared cipher suites ......................................................................................................................... 21 Errors running as SSL server .......................................................................................................................... 22 Verification of server certificate failed ................................................................................................. 22 No shared cipher suites ......................................................................................................................... 22 Plain HTTP client connects to HTTPS Server ...................................................................................... 23 Plain TCP/IP client connects to SSL server .......................................................................................... 23

HPE NonStop cF SSL Library Reference Manual

4

Preface

What is NonStop cF SSL Library NonStop cF SSL Library is a SSL library which brings the power of comForte's SSL implementation to your existing application on the NonStop platform. After replacing some calls in your application source code with the library's SSL-enabled counterpart and by adding a few lines of code for initialization of the library, NonStop cF SSL Library will SSL-enable your existing application. NonStop cF SSL Library builds upon comForte's proven SSL implementation used in other products, such as HPE NonStop SSL.

Understanding the Use of SSL and TLS The use of the words "SSL" and "TLS" can lead to confusion as they are sometimes used interchangeably and sometimes not. Precisely technically speaking, SSL is the predecessor protocol of TLS and started out with SSL version 2.0. The protocol evolved as protocol design vulnerabilities were found and corrected. The successor of SSL 2.0 was SSL 3.0, followed by SSL version 3.1. With 3.1 the SSL protocol had developed to a point where it became clear, that the design wasn't flexible and extensible enough anymore to cope with future requirements. As a result of these limitations, the TLS protocol was developed and introduced. The TLS protocol was designed with backwards compatibility to the SSL protocol in mind and so its original version TLS 1.0 behaves basically in exactly the same way that SSL version 3.1 does. Therefore a notation of SSL 3.1 = TLS 1.0 is commonly found and SSL 3.1 and TLS 1.0 are referred to interchangeably. The SSL protocol was not further enhanced and so the last SSL protocol version is and will remain 3.1. All new enhancement efforts concentrate on extending the TLS protocol nowadays. The TLS protocol has been enhanced in subsequent versions to 1.0 and now TLS 1.1 and TLS 1.2 are standardized. At the time of writing TLS 1.3 is being developed as a new standard, but currently the latest standardized TLS version, TLS 1.2 provides basically everything to create highly secure connections. On the other hand, SSL versions 2.0 and 3.0 are considered as being cryptographically broken and therefore MUST NOT be used anymore. At the time of writing SSL 3.1 / TLS 1.0 are considered to provide good security. For applications where SSL/TLS is to be introduced as a new feature, it is strongly recommend to use TLS 1.2 by default. For applications utilizing earlier versions of TLS, it is strongly recommended to start migrating to TLS 1.2. TLS 1.2 should be the commonly used version by end of year 2016. Speaking about standard industry usage of the terms "SSL" and "TLS", they are used pretty much interchangeably. If not speaking about a specific protocol version, SSL and TLS are assumed to be the same thing, i.e. when people talk/write about "SSL", what they nowadays refer to is basically TLS. Various products, including this product, or also the widely used OpenSSL Library based their names on the original protocol and likely won't change that. However keep in mind that the use of "SSL" in those contexts without a specific version number is referring to both SSL/TLS, respectively in the future to TLS.

Who Should Read This Guide This document is for users of the HPE NonStop cF SSL Library product. As this SSL library is used to add SSL capabilities to applications running on the NonStop platform, the audience will most likely consist of developers making use of NonStop cF SSL Library.

HPE NonStop cF SSL Library Reference Manual

5

Preface

Related Reading NonStop cF SSL Library shares components with other Hewlett Packard Enterprise products; most notably with the component called "SSLOBJ" which is part of HPE NonStop SSL and HPE NonStop cF SSL-AT. This reference manual focuses on the way NonStop cF SSL Library is integrated with your existing code and how this SSL library is configured in general. All details of the Hewlett Packard Enterprise SSL implementation and how to configure it are covered in the •

HPE NonStop SSL Reference Manual

already and are therefore not documented here. Explicit references to the HPE NonStop SSL Reference Manual are made throughout this document.

Document History Version 1.7 The documentation now reflects that Hewlett Packard Enterprise, starting in October 2015, added comForte's original SecurLib/SSL product to the HPE price-book under the product name of NonStop cF SSL Library.

Version 1.6 •

Support for waited sockets included.

•

Section on Installation added.

•

Build instructions update.

•

Change default MINVERSION to 3.1 to match security recommendation.

•

Removed compromised values from default CIPHERSUITES and included additional ones; now: 192.20,192.10,0.57,0.56,0.136,0.135,192.25,192.15,192.5,0.53,0.132,192.18,192.8,0.22,0.19,192.23,192.13 ,192.3,0.10,192.19,192.9,0.51,0.50,0.69,0.68,192.24,192.14,192.4,0.47,0.65,192.17,192.7,192.22,192.12,19 2.2,0.5

•

Add test certificates to package.

•

The function SSL_SetParam can now change the following parameters as well: SERVCERT, SERVKEY, LOGLEVELFILE, LOGLEVELEMS, LOGEMS, LOGLEVELCONSOLE, LOGCONSOLE.

•

The parameter LOGLEVEL has been deprecated.

Version 1.5 •

Support for the FILE_COMPLETE_ API included.

Version 1.4 •

The manual documents SecurLib/SSL version 0016 and later

•

The manual was renamed to “SecurLib/SSL Reference Manual”

•

The function SSL_SetParam can now change the following parameters as well: CLIENTKEYPASS, CLIENTCERT, CLIENTKEY, LOGLEVEL, LOGFILE

Version 1.3 The new function SSL_SetParam has been added.

Version 1.2 The manual layout has been changed.

Version 1.1 This is the initial version of the manual.

HPE NonStop cF SSL Library Reference Manual

6

Using NonStop cF SSL Library

Overview NonStop cF SSL Library works by replacing some TCP/IP and system calls in your application with its own calls. The library calls are named identically except for a prefix SSL_; thus AWAITIOX is replaced with a call to SSL_AWAITIOX, send_nw is replaced with a call to SSL_send_nw and so on. Other than having to change those calls in your source code and having to add some initialization calls no more modifications to your source code are necessary.

Example Socket code prior to using the NonStop cF SSL Library The following diagram (taken from HPE NonStop TCP/IP Programming Manual) shows how a normal nowaited TCP/IP application on the NonStop platform issues calls to the socket calls and to the AWAITIOX routine: Client

Server

1. Optionally, set NonStop TCP/IP or TCP6SAM process name (socket_set_1net_name)

1. Optionally, set NonStop TCP/IP or TCP6SAM process name (socket_set_1net_name)

2. Create a socket (socket_nw, followed by AWAITIOX).

2. Create a socket (socket_nw, followed by AWAITIOX).

3. Optionally, bind the socket to any port (bind_nw, followed by AWAITIOX).

3. Optionally, bind the socket to any port (bind_nw, followed by AWAITIOX).

4. Connect the socket to the server (connect_nw, followed by AWAITIOX)

4. Listen for connections (listen). 5. Accept the connection. a. Accept an incoming connection (accept_nw, followed by AWAITIOX) b. Create a new socket (socket_nw) with (flags & 0200) nowait set. c. Call AWAITIOX, followed by SETMODE 30, followed by AWAITIOX d. Accept the new connection on the new socket (accept_nw2, followed by AWAITIOX).

5. Start data transfer (send_nw and/or recv_nw, followed by AWAITIOX, usually in a loop).

6. Start data transfer on the new socket (recv_nw and/or send_nw, followed by AWAITIOX, usually in a loop).

6. Optionally, shut down the socket (shutdown_nw, followed by AWAITIOX).

7. Optionally, shut down one or both sockets (shutdown_nw, followed by AWAITIOX).

7. Close the socket (CLOSE or FILE_CLOSE_).

8. Close the socket (CLOSE or FILE_CLOSE_).

Table: TCP – Nowait Client and Server Steps

After having changed the application to make use of the NonStop cF SSL Library, the sequence of code will be as shown in the following two sections.

HPE NonStop cF SSL Library Reference Manual

7

Using NonStop cF SSL Library

Client Example Code using NonStop cF SSL Library Note: Changes for usage of the SSL library are highlighted in yellow. 0. Initialize the library SSL_init(config_file_name, license_file_name, 2, SSL_CLIENT)

1. Optionally, set NonStop TCP/IP or TCP6SAM process name (socket_set_inet_name).

2. Create a socket (SSL_socket_nw, followed by SSL_AWAITIOX).

3. Setmode for any io completion order (SSL_SETMODENOWAIT, followed by SSL_AWAITIOX)

4. Optionally, bind the socket to any port (SSL_bind_nw, followed by SSL_AWAITIOX).

5. Connect the socket to the server (connect_nw, followed by SSL_AWAITIOX).

5a. Call SSL_start to make the library aware this is a client socket SSL_start(socket_fd, SSL_CLIENT)

6. Start data transfer (SSL_send_nw and/or SSL_recv_nw, followed by SSL_AWAITIOX, usually in a loop).

7. Optionally, shut down the socket (SSL_shutdown_nw, followed by SSL_AWAITIOX).

8. Close the socket SSL_FILE_CLOSE_ (note that the usage of CLOSE is not supported)

Server Example Code using NonStop cF SSL Library Note: Changes for usage of the SSL library are highlighted in yellow. 0. Initialize the library SSL_init(config_file_name, license_file_name, 2, SSL_SERVER)

1. Optionally, set NonStop TCP/IP or TCP6SAM process name (socket_set_inet_name).

2. Create a socket (SSL_socket_nw, followed by SSL_AWAITIOX).

3. Bind the socket to a well-known port (SSL_bind_nw, followed by SSL_AWAITIOX).

4. Listen for connections (listen).

5. Accept the connection. a. Accept an incoming connection (SSL_accept_nw, followed by SSL_AWAITIOX).

b. Create a new socket with (flags & 0200) nowait set. (SSL_socket_nw)

c. Call

HPE NonStop cF SSL Library Reference Manual

8

Using NonStop cF SSL Library

(SSL_AWAITIOX, followed by SSL_SETMODENOWAIT 30, followed by SSL_AWAITIOX).

d. Accept the new connection on the new socket (SSL_accept_nw2, followed by SSL_AWAITIOX).

e. Call SSL_start to make the library aware this is a server socket SSL_start(socket_fd, SSL_SERVER)

6. Start data transfer on the new socket (SSL_recv_nw and/or SSL_send_nw, followed by SSL_AWAITIOX, usually in a loop).

7. Optionally, shut down one or both sockets (SSL_shutdown_nw, followed by SSL_AWAITIOX).

8. Close the socket SSL_FILE_CLOSE_ (note that the usage of CLOSE is not supported)

Pass-Through Mode NonStop cF SSL Library can be configured to run in so-called Pass-Through mode. In Pass-Through Mode, the library will ***not*** add SSL functionality to the calling application. It will instead transparently pass through all calls to their original counterparts. Pass-Through Mode can be configured if there are any problems with SSL which cannot be resolved. By switching Pass-Through-Mode on, the library will not have any effect whatsoever on the calling application. Note that for pass-through mode to work, SSL needs to be turned off in the TCP/IP application which communicates with your TCP/IP application (running on the NonStop platform) as well.

Step by step guide for adding of NonStop cF SSL Library NonStop cF SSL Library should be added to your existing source code by following the steps outlined below.

Install NonStop cF SSL Library files The NonStop cF SSL Library software is delivered in a ZIP file containing the HPE NonStop_cF_SSL-Lib folder with the following structure: certs

contains test certificates and configuration file

inc

contains include files

lib

contains library file to be bound with your application, in a subdirectory based on NonStop system type and C++ version - the subdirectory SSL_TNSR_v2_Neutral for S-Series systems, C++ version 2 - the subdirectory SSL_TNSR_v3_Neutral for S-Series systems, C++ version 3 - the subdirectory SSL_TNSE_v2_Neutral for Itanium systems, C++ version 2 - the subdirectory SSL_TNSE_v3_Neutral for Itanium systems, C++ version 3 - the subdirectory SSL_TNSX_v2_Neutral for NonStop X systems, C++ version 2 - the subdirectory SSL_TNSX_v3_Neutral for NonStop X systems, C++ version 3

openssl_shell

contains openssl shell for Guardian as type 700, 800 and 500 object file

sample_apps

contains sample application

Transfer the header file SecurLib_ssl.h from the PC location to the location where you are building your application. Make sure the file is transferred in ascii (text) mode. If you are building in the Guardian environment you will need to use a shorter filename, for example comfsslh. Choose the lib subdirectory which matches your machine and C++ version requirements. If you are only using C you may use either version 2 or version 3. Transfer the file seclib.a to the location where you are building your

HPE NonStop cF SSL Library Reference Manual

9

Using NonStop cF SSL Library

application. Make sure the file is transferred in binary mode. If you are building in the Guardian environment you will need to choose a name without an extension, for example secliba. If you wish to use the default certificates transfer them from the PC location to the host location. Make sure CACERT, SERVCERT and SERVKEY are transferred in binary mode, and SSLCFG is transferred in ascii mode.

Include library header file Add the following #include statement to all source files who make socket calls or who call AWAITIOX: #include "comfssl.h"

Initialize the library once Call the initialization routine of the library once during your startup code: int result; /* replace filename below with a zero-terminated string pointing to your configuration file */ char *config_file="$DATA1.SSL.SSLCFG"; char *license_file="$DATA1.SSL.LICENSE"; result=SSL_init(config_file,license_file,2);

Note that the variable result will contain 0 on successful startup and if SSL is activated. If a value of non-zero is returned, NonStop cF SSL Library will run in "pass-through" mode.

Replace socket and nowait calls with calls to the library Edit all your source files containing socket calls or nowait IO calls by replacing the following routines: /* socket calls */ accept[_nw] - note accept becomes SSL_accept1 accept[_nw1] accept2[_nw2] bind[_nw] connect[_nw] - note connect becomes SSL_connect1 getpeername[_nw] getpeername[_nw] getsockname[_nw] getsockname[_nw] getsockopt[_nw] getsockopt[_nw] recv[_nw] send[_nw] send[_nw2] setsockopt[_nw] shutdown[_nw] - note shutdown becomes SSL_shutdown1 socket[_nw] socket_get_len socket_ioctl[_nw] SETMODE[NOWAIT] /* nowait completion calls */ AWAITIOX CANCELREQ FILE_COMPLETE_ /* closing of a socket */ FILE_CLOSE_

with the same routine preceded by the prefix SSL. For example, change result=send_nw(socket,buf,len,0,tag);

to result=SSL_send_nw(socket,buf,len,0,tag);

If the nowait depth in socket_nw is less than 2 change it to 2. This nowait depth is necessary for the SSL handshake to be completed. If is not a requirement to use a nowait open on the socket (flags & 0200), a waited open is acceptable.

HPE NonStop cF SSL Library Reference Manual

10

Using NonStop cF SSL Library

Activate SSL for a given socket Call the following routine once after a socket has been fully connected to activate SSL for that socket: int result; int mode=SSL_SERVER; // or SSL_CLIENT, depending on the // SSL nature of the socket result=SSL_start(socket, mode);

The call SSL_start has to be made before the first SSL_send_nw or SSL_recv_nw is called for that socket. Note: You can call the routine SSL_start multiple times without causing problems. Depending on the logic of your application, it may be easiest to add a call to SSL_start before each call to recv_nw or send_nw.

Error handling Call the following routine after an error has occurred on a socket: long ssl_error; ssl_error=SSL_get_last_error_code(socket);

The routine will return an error number detailing the error. It will return 0 if the error returned is a file-system error which should be retrieved by calling FILE_GETINFO_, otherwise it will return one of the error numbers listed in the appendix. See the function reference for a description of the routine SSL_get_last_error_text which returns a more detailed textual description of the last error.

Bind in the library Add the library to the bind script for your application. Depending on your existing bind script you may need to add additional HPE libraries required by NonStop cF SSL Library. Guardian based examples for each of the system type and version number combinations are below. S-series, version 3 nld $system.system.crtlmain myobject $build.cfssllib.secliba -o myexe -l zstlsrl -obey $system.system.libcobey

S-series, version 2 nld $system.system.crtlmain myobject $build.cfssllib.secliba -o myexe -l zrwslsrl -l zcplsrl -obey $system.system.libcobey

Itanium, version 3 eld $system.system.ccplmain myobject $build.cfssllib.secliba -o myexe -l zcppcdll -l zcpp3dll -obey $system.system.libctxt

Itanium, version 2 eld $system.system.ccplmain myobject $build.cfssllib.secliba -o myexe -l zcppcdll -l zcpp2dll -obey $system.system.libctxt

NonStop X, version 3 xld $system.system.cppmainx $build.cfssllib.secliba -o myexe -l zcppcdll -l zcpp3dll

NonStop X, version 2 xld $system.system.cppmainx $build.cfssllib.secliba -o myexe -l zcppcdll -l zcpp2dll

After following these steps you will have an executable which will provide SSL functionality on its sockets.

HPE NonStop cF SSL Library Reference Manual

11

Using NonStop cF SSL Library

Configuration of NonStop cF SSL Library Configuration Overview NonStop cF SSL Library can be configured in two manners: •

Through a configuration file, the filename of which being passed to the library when calling SSL_init.

•

Through PARAMs which the library reads from the environment.

NonStop cF SSL Library shares most of its configuration value with the SSLOBJ component mentioned in section "Related Reading". Those settings are described in the HPE NonStop SSL Reference Manual. Note: PARAMs take precedence over entries in the configuration file.

Pass-Through Mode The parameter USESSL and the way SSL_Init is called determine if the library runs in pass-through mode. For details see function reference SSL_init.

NonStop cF SSL Library Parameter Reference This section describes all available parameters in alphabetical order. For detailed meaning of a parameter please refer to the HPE NonStop SSL Reference Manual. Note that SSLOBJ has more parameters than listed here; those additional parameters are not relevant to the SSL library and will be ignored. Parameter

Meaning

ALLOWCERTERRORS

allows selective overriding of certificate validation errors. Default "0".

CACERTS

file names of a DER/PEM encoded X.509 CA certificates representing a certificate chain signing the certificate configured with the CLIENTCERT or SERVERCERT parameter. Default "CACERT".

CIPHERSUITES

list of cipher suites that will be accepted by a process utilizing NonStop cF SSL Library. Please see the HPE NonStop SSL Reference Manual for a detailed description of all supported ciphersuites as well as the default ciphersuites used.

CLIENTAUTH

Enforces client authentication when running as SSL server; set of root CA / intermediate CA certificates which sign trusted client certificates.

CLIENTCERT

file name of a DER/PEM encoded X.509 client certificate. Default "*".

CLIENTKEY

the private key to be used for the client certificate. PKCS#8 encoded DER/PEM. Default "*".

CLIENTKEYPASS

password for reading the (encrypted) private key file. Default "test".

LICENSE

the location for the license file of NonStop cF SSL Library. Default "LICENSE".

LOGCONSOLE

determines if log messages are written to a console. Default "*" (no logging).

LOGEMS

determines it log messages are written to EMS. Default "*" (no logging).

LOGFILE

determines if log messages are written to a file. Default "*" (no logging).

LOGFORMATCONSOLE

controls the format of the log messages written to the console. Default "93".

LOGFORMATEMS

controls the format of the log messages written to EMS. Default "64".

LOGFORMATFILE

controls the format of the log messages written to the log file. Default "93".

LOGLEVELCONSOLE

determines which messages are written to the console. Default "50".

LOGLEVELEMS

determines which messages are written to EMS. Default "50".

LOGLEVELFILE

determines which messages are written to the log file. Default "50".

LOGMAXFILELENGTH

controls the maximum size of the log file. Default "20000".

MAXVERSION

maximum admissible SSL/TLS protocol version. Default "3.3" (=TLS 1.2).

MINVERSION

minimum admissible SSL/TLS protocol version. Default "3.1" (=SSL 3.1 = TLS 1.0).

SERVCERT

file name of a DER/PEM encoded X.509 server certificate. Default "SERVCERT".

HPE NonStop cF SSL Library Reference Manual

12

Using NonStop cF SSL Library

Parameter

Meaning

SERVKEY

the private key to be used for the server certificate. PKCS#8 encoded DER/PEM. Default "SERVKEY".

SERVKEYPASS

password for reading the (encrypted) private key file. Default "test".

TRUST

When running as SSL client: list of trusted CA or server certificate files or fingerprints. When running as SSL server - use PARAM CLIENTAUTH instead. Default "*"

USESSL

Can be used to switch SSL on or off through the configuration file or PARAMs. Default "1".

Licensing NonStop cF SSL Library is licensed through a license file delivered by Hewlett Packard Enterprise. The location of the license file must be supplied to the call of SSL_init. If the license check fails, NonStop cF SSL Library will run in pass-through mode only.

HPE NonStop cF SSL Library Reference Manual

13

Function Reference

Replacement of existing calls The following calls within NonStop cF SSL Library are replacement of existing calls. All the routines have the exact same syntax and semantics as their original counterpart, see the HPE NonStop documentation for details. Note that the functions accept, connect and shutdown are replaced by SSL_accept1, SSL_connect1 and SSL_shutdown1 respectively. If you are using waited sockets you must use the replacement functions for all of your existing calls. If you are using nowait sockets some of the functions below are new and it is not absolutely necessary to use them. However this is for backwards compatibility and it is recommended that you use the replacement functions when writing new code. The functions that are not strictly required are: SSL_accept_nw, SSL_accept_nw1, SSL_accept_nw2, SSL_bind_nw, SSL_connect_nw, SSL_socket_nw, SSL_SETMODENOWAIT.

SSL_accept1 int SSL_accept1 (int, struct sockaddr *, int *)

SSL_accept_nw int SSL_accept_nw (int, struct sockaddr *, int *, long)

SSL_accept_nw1 int SSL_accept_nw1 (int, struct sockaddr *, int *, long, int)

SSL_accept_nw2 int SSL_accept_nw2 (int, struct sockaddr *, long)

SSL_bind int SSL_bind (int, struct sockaddr *, int)

SSL_bind_nw int SSL_bind_nw (int, struct sockaddr *, int, long)

SSL_connect_nw int SSL_connect_nw (int, struct sockaddr *, int, long)

HPE NonStop cF SSL Library Reference Manual

14

Function Reference

SSL_connect1 int SSL_connect1 (int, struct sockaddr *, int)

SSL_connect_nw int SSL_connect_nw (int, struct sockaddr *, int, long)

SSL_getpeername int SSL_getpeername (int ,struct sockaddr *,int *)

SSL_getpeername_nw int SSL_getpeername_nw (int ,struct sockaddr *,int *,long)

SSL_getsockname int SSL_getsockname (int, struct sockaddr *, int *)

SSL_getsockname_nw int SSL_getsockname_nw (int, struct sockaddr *, int *, long)

SSL_getsockopt int SSL_getsockopt (int , int , int optname, char *, int *)

SSL_getsockopt_nw int SSL_getsockopt_nw (int , int , int optname, char *, int *, long)

SSL_recv int SSL_recv (int, char *, int, int)

SSL_recv_nw int SSL_recv_nw (int, char *, int, int, long)

SSL_send int SSL_send (int, char *, int, int)

SSL_send_nw int SSL_send_nw2 (int, char *, int, int, long)

SSL_send_nw2 int SSL_send_nw2 (int, char *, int, int, long)

HPE NonStop cF SSL Library Reference Manual

15

Function Reference

SSL_setsockopt int SSL_setsockopt (int , int , int , char *, int)

SSL_setsockopt_nw int SSL_setsockopt_nw (int , int , int , char *, int , long)

SSL_shutdown1 int SSL_shutdown1(int, int)

SSL_shutdown_nw int SSL_shutdown_nw (int, int, long)

SSL_socket int SSL_socket (int, int, int)

SSL_socket_nw int SSL_socket (int, int, int, int, int)

SSL_socket_get_len int SSL_socket_get_len (int)

SSL_socket_ioctl int SSL_socket_ioctl (int , int , char *)

SSL_socket_ioctl_nw int SSL_socket_ioctl_nw (int , int , char *, long)

SSL_AWAITIOX _extensible _cc_status SSL_AWAITIOX (short _far *, long _far *, short _far *, long _far *, long, short _far *)

SSL_CANCELREQ _extensible _cc_status SSL_CANCELREQ(short, long)

SSL_FILE_CLOSE_ _extensible short SSL_FILE_CLOSE_ (short, short)

SSL_FILE_COMPLETE_ _extensible _cc_status SSL_FILE_COMPLETE_(short _far *completion_info, __int32_t timelimit, short _far *complete_element_list, short num_complete_elements, short _far

HPE NonStop cF SSL Library Reference Manual

16

Function Reference

*error_complete_element);

SSL_SETMODE _variable _cc_status SSL_SETMODE(short, short, short, short, short _near *)

SSL_SETMODENOWAIT _variable _cc_status SSL_SETMODENOWAIT(short, short, short, short, short _near *, long) (note to writer: SSL_sock_close_reuse_nw has been deliberately removed)

Calls added by NonStop cF SSL Library NonStop cF SSL Library provides a couple of additional routines to be called by your application code. This section explains the syntax and semantic of those calls.

SSL_init int SSL_init( char * cfg_file, char * license_file, int use_ssl_mode, int ssl_client_or_server )

This routine reads SSL parameters and initializes the library. It should be called one time during startup of your program. It will return 0 only if SSL will be turned on.

Parameters: Parameters

Meaning

cfg_file

name of config file or NULL if no config file is available

license_file

name of license file

mode

determines if SSL will be turned on or off:

mode=0

Value of USESSL will be ignored, SSL will be turned OFF

mode=1

Value of USESSL will be ignored, SSL will be turned ON

mode=2

Value of USESSL will be used to determine if SSL is turned on or off

ssl_client_or_server

determines if the library will work as SSL client or server (or both).

SSL_SERVER

to only run as SSL server,

SSL_CLIENT

to only run as SSL client

(SSL_SERVER | SSL_CLIENT)

to run as both

Returns: Returns

Meaning

0

SSL is turned on

-1

SSL is turned off by parameter "mode" passed as 0

-2

SSL is turned off because "mode" was 2 and USESSL was 0

-3

SSL is turned off because the license check failed

-4

error initializing SSL, SSL is turned off

-5

SSL is turned off because there was an error reading certificates.

(File system error larger than zero)

configuration file was passed but could not be read; SSL is turned off. The file system error is returned.

HPE NonStop cF SSL Library Reference Manual

17

Function Reference

SSL_start int SSL_start( int socket, int mode )

This routine starts SSL for a given socket in the given mode. Call this function before issuing the first call to SSL_send, SSL_send_nw, SSL_send_nw2, SSL_recv or SSL_recv_nw on a socket.

Parameters: Parameters

Meaning

socket

socket file number

mode

SSL_SERVER or SSL_CLIENT

Returns: Returns

Meaning

0

no error

error

a problem occurred

Considerations: This call should always return 0. If it does not, there either is an unexpected problem in NonStop cF SSL Library or there is an error in the way you are calling the library routines.

SSL_SetParam char *SSL_SetParam(char *param, char *value);

This routine sets configuration parameters programmatically. It has to be called before calling SSL_init.

Parameters: Parameters

Meaning

param

the name of the parameter

value

the value of the parameter

Returns: Returns

Meaning

NULL

no error

a zero-terminated string describing the error

a problem occurred

Considerations: The following parameter can be changed using SetParam: •

SERVKEYPASS, CLIENTKEYPASS, CLIENTCERT, CLIENTKEY, CACERTS, SERVCERT, SERVKEY, LOGLEVELCONSOLE, LOGLEVELEMS, LOGLEVELFILE, LOGEMS, LOGCONSOLE, LOGFILE

•

The parameters LOGLEVELCONSOLE, LOGLEVELEMS, LOGLEVELFILE, LOGEMS, LOGCONSOLE and LOGFILE can be changed either before or after SSL_init has been called. The other parameters can only be changed before SSL_init has been called.

•

For the meaning of the parameters please see section "Parameter_Reference " and/or the "HPE NonStop SSL Reference Manual".

HPE NonStop cF SSL Library Reference Manual

18

Function Reference

SSL_get_encryption_method char* SSL_get_encryption_method( int socket )

This routine retrieves the current encryption method for a given socket.

Parameters: Parameters

Meaning

socket

socket number

Returns: Returns

Meaning

char*

contains the string representation of encryption method

NULL

if no encryption method is active on that socket.

Considerations: NULL can be returned for the following reasons: •

NonStop cF SSL Library is in "pass-through-mode".

•

The SSL handshake on that socket has not completed yet.

SSL_get_last_error_code int SSL_get_last_error_code(int socket)

This routine determines the last SSL error number generated on a given socket.

Parameters: Parameters

Meaning

socket

socket number

Returns: Returns

Meaning

0

no SSL error occurred

SSL error number File System error number

Considerations: If a socket routine returned an error through AWAITIOX you should retrieve the file system error on that socket with a call to FILE_GETINFO. If that yields no file system error, you should retrieve the SSL error number with a call to SSL_get_last_error_code. You can also retrieve a detailed textual description of the last error within the SSL library by calling SSL_get_last_error_text. See chapter "Error numbers and Error texts" for details on the error numbers.

SSL_get_last_error_text char *SSL_get_last_error_code()

This routine returns a textual representation of the last problem occurring within the library.

Parameters: (none)

HPE NonStop cF SSL Library Reference Manual

19

Function Reference

Returns: Textual representation of last error occurred within the library. The textual representation is returned as a "C-type" zero-terminated string. See chapter "Error numbers and Error texts" for some examples.

Considerations: In contrast to SSL_get_last_error_code, this call returns the last error text for all sockets.

HPE NonStop cF SSL Library Reference Manual

20

Error numbers and Error texts within NonStop cF SSL Library

Introduction SSL is a complex protocol. Therefore many things can go wrong resulting in many potential errors. Within this appendix, some common error situations are explained and the associated error numbers are listed. The error numbers returned by NonStop cF SSL Library are the standard error numbers used within OpenSSL. If an error number occurs which is not listed in here, Hewlett Packard Enterprise will be able to assist in analysis given the error number. Simple calling SSL_get_last_error_text and logging the textual error message somewhere within your application may prove to be sufficient in a lot of scenarios. This chapter is organized into two sections: the first section lists errors running as SSL client, the second one errors running as SSL server. Note that some error scenarios apply to both cases; however the error numbers and messages will be different.

Errors running as SSL client The following errors can occur when NonStop cF SSL Library is used to run as SSL client.

Verification of server certificate failed SSL error number: 0x14090086

Full SSL error text: error code: 0x14090086 in I:\openssl\0.9.7a\work1-Win32\ssl\s3_clnt.c in line 843. error text: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Explanation: In SSL, the clients need to authenticate the server; this happens during SSL session setup. If the client decides he does not trust the server he will issue that error message. Remedy: Check server certificate and client configuration. Within NonStop cF SSL Library, the parameter for the client is TRUST.

No shared cipher suites SSL error number: 0x14077410

HPE NonStop cF SSL Library Reference Manual

21

Error numbers and Error texts within NonStop cF SSL Library

Full SSL error text: error code: 0x14077410 in I:\openssl\0.9.7a\work1-Win32\ssl\s23_clnt.c in line 470. error text: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Explanation: SSL allows configuring so-called cipher-suites both on the client and server. If no matching cipher suites exist between client and server, the server will send an alert and the client will return that error. Remedy: Check client and server configuration. Within NonStop cF SSL Library, cipher suites are configured using the parameter CIPHERSUITES.

Errors running as SSL server The following errors can occur when NonStop cF SSL Library is used to run as SSL server.

Verification of server certificate failed SSL error number: 0x140940e5

Full SSL error text: error code: error data: error text: unknown error code: error text:

0x14094416 in I:\openssl\0.9.7a\work1-Win32\ssl\s3_pkt.c in line 1052. SSL alert number 46 error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate 0x140940e5 in I:\openssl\0.9.7a\work1-Win32\ssl\s3_pkt.c in line 837. error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure

Explanation: In SSL, the clients need to authenticate the server; this happens during SSL session setup. If the client decides he does not trust the server he will send an alert to the server which will in turn report that error number and message. Remedy: Check server certificate and client configuration. Within NonStop cF SSL Library, the parameters for the server are SERVKEY, SERVCERT and CACERTS.

No shared cipher suites SSL error number: 0x1408a0c1

Full SSL error text: error code: 0x1408a0c1 in I:\openssl\0.9.7a\work1-Win32\ssl\s3_srvr.c in line 881. error text: error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher

Explanation: SSL allows configuring so-called cipher-suites both on the client and server. If no matching cipher suites exist between client and server, the server will report that error. Remedy: Check client and server configuration. Within NonStop cF SSL Library, cipher suites are configured using the parameter CIPHERSUITES.

HPE NonStop cF SSL Library Reference Manual

22

Error numbers and Error texts within NonStop cF SSL Library

Plain HTTP client connects to HTTPS Server SSL error number: 0x1407609c

Full SSL error text: error code: 0x1407609c in I:\openssl\0.9.7a\work1-Win32\ssl\s23_srvr.c in line 400. error text: error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request

Explanation: If a new SSL connection is established, the SSL client and server set up the session following a specific protocol. If a normal HTTP requests arrives as the first packet from the SSL client, the SSL server will issue that specific error. Remedy: Make sure the client uses SSL as well.

Plain TCP/IP client connects to SSL server SSL error number: 0x140760fc

Full SSL error text: error code: 0x140760fc in I:\openssl\0.9.7a\work1-Win32\ssl\s23_srvr.c in line 585. error text: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol

Explanation: If a new SSL connection is established, the SSL client and server set up the session following a specific protocol. If the first packet from the SSL client, the SSL server does not follow that protocol, the server will report that error. Remedy: Make sure the client uses SSL as well.

HPE NonStop cF SSL Library Reference Manual

23