DENODO PLATFORM 6.0 INSTALLATION GUIDE
Update Dec 21st, 2016 Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
The present software, along with any documentation and fonts accompanying this License, is owned by Denodo Technologies. All intellectual property rights belong to Denodo Technologies or our suppliers. The user of software is expressly committed to respect the intellectual property rights owned by Denodo Technologies according to the terms of the granted license of use, as well as to what is established in the laws for protection of intellectual property in force at any time, both nationally and internationally. The user also declares to know the terms of the granted license of use and expressly accepts all of them. This document is confidential and is the property of Denodo Technologies. No part of this document may be reproduced in any form by any means without prior written authorization from Denodo Technologies. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/) This product includes (
[email protected])
cryptographic
software
written
by
Eric
Young
This product includes software written by Tim Hudson (
[email protected])
Copyright 2016 Denodo Technologies, Inc. Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
Denodo Platform 6.0
Installation Guide
CONTENTS CONTENTS ................................................................................................... I LIST OF FIGURES ......................................................................................... I LIST OF TABLES ........................................................................................... I PREFACE .................................................................................................... II SCOPE II WHO SHOULD USE THIS DOCUMENT ...................................................... II SUMMARY OF CONTENTS ....................................................................... II 1
INTRODUCTION .......................................................................... 1
2
PREINSTALLATION TASKS .......................................................... 3 2.1 UPGRADING FROM A PREVIOUS VERSION .................................. 3 2.2 CHECKING THE HARDWARE REQUIREMENTS .............................. 3 2.3 CHECKING THE SOFTWARE REQUIREMENTS ............................... 4 2.3.1 Software Requirements for Virtual DataPort ...................................... 5 2.3.2 Software Requirements for ITPilot .................................................... 5 2.3.3 Software Requirements for Aracne ................................................... 6 2.3.4 Supported Browsers ....................................................................... 6 2.4 OTHER PRE-INSTALLATION TASKS ............................................. 6 2.4.1 Check that the Required Ports are Free ............................................. 6 2.4.2 Check the PATH Environment Variable on Windows ............................ 6 2.4.3 Select a User Account to Install the Denodo Platform ......................... 7 2.4.4 Upgrading from a Previous Version .................................................. 7 2.5 DOWNLOAD AN INSTALLER ........................................................ 7
3
USING THE GRAPHICAL INSTALLATION WIZARD ........................ 9 3.1 COMMONS SETTINGS ................................................................ 13 3.2 COMPONENTS AND CONFIGURATION OF VIRTUAL DATAPORT .. 14 3.3 COMPONENTS AND CONFIGURATION OF ITPILOT..................... 15 3.3.1 Initial Configuration of the Browser Pool ..........................................16 3.3.2 Wrapper Server ............................................................................16 3.3.3 Verification Server ........................................................................17 3.4 ARACNE .................................................................................... 17 3.4.1 Aracne Server ..............................................................................18 3.4.2 Indexing/Search Engine Server ......................................................18 3.5 SCHEDULER............................................................................... 19 3.6 EMBEDDED WEB CONTAINER .................................................... 19
4
USING THE COMMAND LINE INSTALLER ................................... 20
5
POST-INSTALLATION TASKS ..................................................... 22 INSTALL THE LICENSE .............................................................. 22 INSTALL THE LATEST UPDATE ................................................... 22 ENABLE SSL CONNECTIONS IN THE DENODO PLATFORM SERVERS 22 5.3.1 Obtaining and Installing an SSL Certificate .......................................23 5.3.2 Enabling SSL in Denodo Platform Servers ........................................25
5.1 5.2 5.3
Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
Denodo Platform 6.0
Installation Guide
5.3.3 Enabling HTTPS in the Embedded Apache Tomcat .............................26 5.3.4 Enabling SSL in Denodo Platform Tools ............................................28 5.3.5 Enabling SSL for External Clients ....................................................29 5.4 CONFIGURE WINDOWS SERVICES ............................................ 30 5.5 POST-INSTALLATION TASKS IN VIRTUAL DATAPORT ............... 30 5.5.1 Change the Default Password .........................................................30 5.5.2 Preparing the Connection to Databases............................................31 5.5.3 Installing the SAP JCo Connector ....................................................31 5.5.4 Grant Privileges in SAP BW to the Virtual DataPort User Account .........34 5.5.5 Grant Privileges on SAP for BAPI Sources .........................................35 5.5.6 Installing the Connector for SAP BW and SAP BI (Multidimensional Sources) ......................................................................................35 5.5.7 Installing the Connector for Oracle Essbase ......................................36 5.5.8 Enable XMLA on Microsoft SQL Server Analysis Services (Multidimensional Sources) ............................................................36 5.5.9 Installing the JMS Connectors to Create JMS Listeners and Web services with SOAP over JMS ..........................................................37 5.5.10 Configuring the VCS Clients ......................................................38 5.5.11 Importing the Certificates of Data Sources (SSL Connections) ......41 5.5.12 Increasing the Maximum Simultaneous Requests ........................42 5.5.13 Setting-up Kerberos Authentication in Virtual DataPort ................43 5.5.14 Setting-up Kerberos Authentication in the Information SelfService Tool .................................................................................47 5.5.15 Enabling the Support for ODBC Sources When the Virtual DataPort Server Runs on Linux .......................................................49 5.5.16 Resource Manager: Modify the Priority of a Query Dynamically .....50 5.5.17 Launching the Administration Tool in High DPI Displays ...............50 5.6 POST-INSTALLATION TASKS IN ITPILOT .................................. 50 5.6.1 Checking that the Generation Environment Has Been Installed ...........50 5.6.2 Configuration of Microsoft Internet Explorer .....................................51 5.6.3 Enabling Internet Explorer Sequence Generation Toolbar in Microsoft Windows Server 2008 ......................................................52 5.6.4 Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows 2008 ................................................................52 5.6.5 Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows Server 2012 ......................................................53 5.6.6 Manual Configuration of Adobe Acrobat Professional ..........................53 5.6.7 Automatic Verification Database .....................................................53 5.6.8 Launching the Wrapper Generator Tool in High DPI Displays ..............54 5.7 POST-INSTALLATION TASKS IN ARACNE AND SCHEDULER ....... 54 5.7.1 Microsoft Internet Explorer Configuration .........................................54 6
UNINSTALLATION ..................................................................... 55
7
DENODO4E ................................................................................ 56
8
DENODO PLATFORM CONTROL CENTER ..................................... 57 8.1 EXECUTION ............................................................................... 57 8.2 CONTROL CENTER HELP ............................................................ 57 8.2.1 Online Help Configuration ..............................................................59 8.3 STARTING PLATFORM SERVERS AND TOOLS ............................. 60 8.3.1 Configuration of Custom Environments ............................................61 8.4 INSTALLING UPDATES AND HOTFIXES...................................... 63 8.5 VIRTUAL MACHINE AND WEB CONTAINER CONFIGURATION .... 64
Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
Denodo Platform 6.0
8.6 8.7 LINE
Installation Guide
UNINSTALLING THE PLATFORM ................................................ 65 CONFIGURATION OF THE JVM PARAMETERS FROM THE COMMAND 67
9 UNATTENDED INSTALLATION OF THE DENODO PLATFORM AND ITS UPDATES ............................................................................................ 68 9.1 MODIFYING THE DENODO PLATFORM INSTALLER TO INCLUDE THE LAST UPDATE ........................................................................................ 68 9.2 UNATTENDED INSTALLATION OF THE DENODO PLATFORM ....... 68 9.3 UNATTENDED INSTALLATION OF UPDATES AND HOTFIXES ...... 70 10 INSTALLING THE DENODO WEB APPLICATIONS AS WINDOWS SERVICES 71 11 APPENDIX ................................................................................. 72 11.1 DEFAULT PORTS USED BY THE DENODO PLATFORM MODULES .. 72 11.2 LIMITATIONS OF THE DENODO EXPRESS LICENSE ................... 74 11.3 USING KERBEROS AUTHENTICATION IN VIRTUAL DATAPORT WITHOUT JOINING A KERBEROS REALM .............................................. 75 11.4 USING KERBEROS AUTHENTICATION IN THE INFORMATION SELFSERVICE TOOL WITHOUT JOINING A KERBEROS REALM ...................... 76 11.5 PROVIDING A KRB5 FILE FOR KERBEROS AUTHENTICATION .... 78 11.6 CONFIGURING INTERNET EXPLORER RUNNING UNDER THE LOCAL SYSTEM ACCOUNT................................................................................. 80 11.7 LAUNCHING THE DENODO STANDALONE APPLICATIONS IN HIGH DPI DISPLAYS ...................................................................................... 81 11.8 DEPLOYING THE WEB ADMINISTRATION TOOL IN AN EXTERNAL WEB CONTAINER .................................................................................. 83 REFERENCES ............................................................................................. 85
Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
Denodo Platform 6.0
Installation Guide
LIST OF FIGURES Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
General architecture of the Denodo Platform ..................................... 2 Administrator credentials request dialog ........................................... 9 Initial installation screen for the Denodo Platform .............................10 Step 2: selecting the installation method .........................................11 Step 3: selecting the modules to install ...........................................12 Summary screen ..........................................................................13 Choosing the internationalization configuration .................................14 SAP Java Connector (JCo) test screen .............................................33 Active Directory User Configuration .................................................45 Syntax of the setspn utility ............................................................45 Example of declaring a Service Principal Name with setspn ................46 Syntax of the ktpass utility ............................................................46 Example of generating a keytab file ................................................46 Configuring the local repository for Denodo 4E .................................56 Denodo Platform Control Center Help screen ....................................58 Information on the current license ..................................................59 Online help configuration dialog ......................................................59 Denodo Platform Control Center .....................................................60 Creation of custom environments....................................................62 Creation and editing of custom environments ...................................63 Uninstalling the Denodo Platform ....................................................65 Processes running warning .............................................................66 Sample krb5 file ...........................................................................78
Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
Denodo Platform 6.0
Installation Guide
LIST OF TABLES Table 1 Table 2 Table 3
Hardware requirements of the Denodo Platform ................................. 3 Properties to modify to change the JVM parameters ..........................67 Commands to install the Denodo web applications as a Windows service 71 Table 4 Default TCP/IP port numbers opened by the Denodo Platform modules 73 Table 5 Limitations of the Denodo Express license (Virtual DataPort and Scheduler) 74 Table 6 Default location of the krb5 file depending on the operating system ....78
Denodo Technologies North America & APAC: 530 Lytton Avenue, Suite 301. Palo Alto, CA 94301. USA Denodo Technologies Iberia & Latinoamérica: C/Montalbán, 5, 28014 Madrid. SPAIN Denodo Technologies EMEA: 19th Floor, Portland House, Bressenden Place, London SW1E 5RS. UK
www.denodo.com
Denodo Platform 6.0
Installation Guide
PREFACE
SCOPE The Denodo Platform provides business applications with easy access to integrated views of various heterogeneous and distributed, both structured and non-structured, data sources. The Platform includes modules for real-time access and integration of all types of information, including that contained in databases, Web Services, hidden Web sites, and document repositories. This document introduces readers to the installation and configuration of the Platform and the use of the Denodo Platform Control Center tool that, among other uses, provides a single panel from where all the servers and tools comprising the Platform can be started and stopped. WHO SHOULD USE THIS DOCUMENT This document is aimed at developers and administrators requiring detailed information on how to install and configure the Denodo Platform and how to use the Denodo Platform Control Center tool. SUMMARY OF CONTENTS More specifically, this document:
Briefly introduces the Denodo Platform.
Describes the processes required to install each of the Platform modules.
Describes the use of the Denodo Platform Control Center that, among other uses, provides a single panel from where all the servers and tools comprising the Platform can be started and stopped.
Preface
i
Denodo Platform 6.0
1
Installation Guide
INTRODUCTION
Modern business processes must make use of a vast array of information systems, developed over several decades. These systems, in turn, make use of a vast array of data sources, developed for the most part using completely disparate technologies (relational databases, Web services, XML documents, spreadsheets, flat files …) and very dissimilar data models. Some of these data sources are "well-structured" for ease of querying from client applications; more and more are less well structured and lack even rudimentary schema for facilitating client interactions. Some, in fact, are pure text documents, which make no pretense at adhering to a "structure" of any kind. Development of modern information systems and optimization of business processes require access and utilization of the data stored in this vast array of inherited data repositories. Another important trend to consider is the expansion of Social Media and Big Data, an obvious example of the widespread dissemination of needed information and the heterogeneity of data formats. This data, typically found through browsing the Web in its many forms, often adds crucial value to corporate information systems. In addition, clients, suppliers, and competitors often use simple or complex Web interfaces for accessing their information and services. The Denodo Platform is a global solution for the real-time integration of heterogeneous and dispersed, structured and non-structured data sources. To do so, it combines different integrated modules (see Figure 1):
The Virtual DataPort module provides real-time integration of any relevant information for the company, whatever its origin, format, and structure level. It enables to create “virtual databases” that are made up of unified views of data from any type of data source, including databases, SOAP, and REST Web Services, XML, flat files, conventional Web sites (through the ITPilot module), LDAP servers, and non-structured information indexes (through the Aracne module).
The ITPilot module provides an easy way to access and structure the data present in the Web. This process involves building an abstraction from the specific Web source called wrapper that isolates the client applications from the intrinsic characteristics of that site (access protocol, native data structure, etc.). Thus, the data contained on the hidden Web can be queried and integrated just as easily and effectively as the data contained in a conventional database. In addition to that, ITPilot can also extract structured data from PDF, MS Excel and MS Word documents. ITPilot provides a distributed and scalable environment for generating, executing, and verifying the correct operation of wrappers.
The Aracne module facilitates crawling, indexing, filtering, and querying of non-structured data in a wide range of repositories and formats. Aracne can integrate information contained in repositories such as the Web, file systems, relational databases, or e-mail servers.
In addition, the Scheduler module is a complementary tool that allows scheduling batch tasks using any of the former modules to automatically
Introduction
1
Denodo Platform 6.0
Installation Guide
extract and filter data from several sources and export them to the desired format.
Figure 1
General architecture of the Denodo Platform
This document introduces readers to the installation and configuration of the Platform and the use of the Denodo Platform Control Center tool that, among other uses, provides a single panel from where all the servers and tools comprising the Platform can be started and stopped. See the Virtual DataPort Administration Guide [VDP], the ITPilot User Guide [ITPILOT], the Aracne Administration Guide [ARCN] and the Scheduler Administration Guide [SCH] for information about how to use each module.
Introduction
2
Denodo Platform 6.0
2
Installation Guide
PREINSTALLATION TASKS
This section describes the tasks that you must complete before starting the Denodo Platform installer. These include:
Checking the hardware requirements of the host: see section 2.1.
Checking the software requirements: see section 2.3.
Other preinstallation tasks: see section 2.4.
2.1
UPGRADING FROM A PREVIOUS VERSION
If you are going to upgrade to the Denodo Platform version 6.0 from a previous version read the Denodo Platform Migration Guide [DENMIGR] before continuing the with installation of the new version. 2.2
CHECKING THE HARDWARE REQUIREMENTS
The Denodo Platform has the following hardware requirements: 2 GHz or higher. Processor
The performance of a processor depends on its clock frequency, but also on its number of cores and the size of the processor cache.
Physical memory (RAM)
4 Gigabytes.
Disk space
Minimum: 2 Gigabytes
When installing the Denodo Platform on a virtual machine, the amount of memory assigned to this virtual machine should be backed by physical memory. That is, the memory should not be shared with other virtual machines. Otherwise, the host operating system will have to swap to disk parts of the virtual machine. This will lead to a severe decrease of the performance of the Denodo Platform.
Recommended: 10 GB Recommended for very complex scenarios involving billions of rows: 50GB. See the explanation below. Table 1
Hardware requirements of the Denodo Platform
The amount of free disk space required depends on the use of the Denodo Platform:
A full installation of the Denodo Platform (i.e. installation of all the modules: Aracne, ITPilot, Scheduler and Virtual DataPort) uses 1.5 gigabytes.
Preinstallation Tasks
3
Denodo Platform 6.0
Installation Guide
The metadata of each module of the Denodo Platform is stored locally (e.g. information about the data sources, views, web services, Scheduler jobs, etc.) and usually does not go over a few hundred megabytes.
Space used by updates: Denodo keeps a backup copy of the libraries that are being updated. Each update takes up to 300 megabytes. Considering the number of updates released per each major version, the backup copies end up using around 5 gigabytes.
Space used for swapping data: to avoid memory overflows, Virtual DataPort swaps to disk the intermediate results of queries when they do not fit in memory. Usually, most of the processing is pushed down to the source so the number of rows processed by Virtual DataPort is low and swapping is not required. However, with very complex queries where Virtual DataPort processes billions of rows, some data is swapped to disk to avoid a memory overflow. This is why, we recommend having 50 gigabytes of free disk space. The section “Memory Management” of the Virtual DataPort Administration Guide explains when Virtual DataPort swaps intermediate results to disk.
2.3
CHECKING THE SOFTWARE REQUIREMENTS
Before installing the Denodo Platform, you need to install the following: 1. An operating system with Java support. The Denodo Platform runs on any of the platforms listed below: o
Windows: Windows Server 2012, Windows Server 2008, Windows 10 (only officially supported for the Virtual DataPort module), Windows 8.1 and Windows 7.
o
Solaris
o
Linux: Ubuntu 12.04 LTS and 14.04 LTS, CentOS 6 and 7, Red Hat Linux 6 and 7, Oracle Linux 6 and 7.
2. Denodo provides a Denodo Platform installer for several environments: 32-bit Windows, 64-bit Windows, 32-bit Linux and 64-bit Linux. These installers include the appropriate Java Runtime Environment (JRE). If you are installing the Denodo Platform in an environment other than these ones, install a Java Runtime Environment version 1.7. For an updated list of the supported platforms, contact the Denodo Support Team. Besides these requirements, each module of the Denodo Platform has additional requirements that must be met in order to use some of their components:
Additional requirements of Virtual DataPort: see section 2.3.1.
Additional requirements of ITPilot: see section 2.3.2.
Preinstallation Tasks
4
Denodo Platform 6.0
Installation Guide
Additional requirements of Aracne: see section 2.3.3.
2.3.1
Software Requirements for Virtual DataPort
This section lists the additional requirements of Virtual DataPort. 2.3.1.1
Microsoft Access Sources
To extract data from Microsoft Access files, you need to install the Access ODBC drivers. To obtain them, do one of the following:
Install Microsoft Office in the same machine as the Virtual DataPort Server
Or, install the “Microsoft Access Database Engine 2010 Redistributable” (http://www.microsoft.com/download/en/details.aspx?id=13255)
2.3.1.2
Cache
Virtual DataPort incorporates a cache module that stores local copies of the data retrieved from the data sources, in a JDBC database. This may reduce the impact of repeated queries hitting the data source and speed up data retrieval, especially with certain type of sources. To be able to cache data, you need to do the following: 1. Install an external Database Management Systems (DBMSs). 2. Create a schema on this database. Denodo will use it to store the cached data. Although Denodo can use a schema used by other applications, we recommend creating a new one so the objects created by Denodo do not conflict with the ones created by other applications. 3. Grant the following privileges to the user account that Denodo will use to connect to this database: o
Privileges to create and drop tables on the schema.
o
Privileges to execute SELECT, INSERT, UPDATE and DELETE statements on these tables.
Important: Virtual DataPort embeds an Apache Derby [DERBY] database that can be used to store the cache data. However, we strongly advise against using it on a production environment. This database is provided just for demoing and development purposes. The section “Cache Module” of the Administration Guide [VDP] explains how the cache module works and lists the DBMSs that Virtual DataPort can use to store the cached data. 2.3.2
Software Requirements for ITPilot
This section lists the additional requirements of ITPilot:
To use of the automatic Browser Pool, install Microsoft Internet Explorer 10.x or 11.x [MSIE].
Preinstallation Tasks
5
Denodo Platform 6.0
Installation Guide
To extract data from PDF files using the Adobe PDF engine, ITPilot has to run on Windows with Adobe Acrobat Professional 9 or 10.
To extract data from Microsoft Word files, install OpenOffice 4.0.x or 4.1.x [OOFF]
The Verification Server uses a relational database to store information about wrappers execution to allow verifying its correct operation. ITPilot provides an embedded Apache Derby [DERBY] database that can be used for this purpose. An external JDBC database management system can also be used. This feature has been tested with MySQL [MYSQL] and Oracle [ORCL]. ITPilot provides a script to create tables for these DB Management Systems (consult the Post-Installation tasks for more details about these scripts).
2.3.3
Software Requirements for Aracne
This section lists the additional requirements of Aracne:
2.3.4
To use Aracne with IECrawler, Aracne has to run on Windows with Microsoft Internet Explorer 10.x or 11.x [MSIE]. Supported Browsers
The Information Self Service Tool, the Diagnostic & Monitoring Tool and the web administration tools are tested with Chrome 49.x, Internet Explorer 11 and Mozilla Firefox 44.x. Although they are tested with these versions, they should work properly with any modern desktop web browser. 2.4
OTHER PRE-INSTALLATION TASKS
This section describes tasks that you must complete before you install the Denodo Platform. 2.4.1
Check that the Required Ports are Free
The servers of the Denodo Platform listen for incoming connections on several ports. The appendix 11.1 lists the default ports. Make sure these ports are available on your system. If they are not available, select others that are free during the installation process. 2.4.2
Check the PATH Environment Variable on Windows
Ignore this subsection if you are installing the Denodo Platform on Linux. If you are installing the Denodo Platform on Windows, check that the PATH environment variable meets these rules: 1. PATH cannot have double quotes. If it does, the modules of the Denodo Platform will not start. E.g. the following PATH is invalid:
Preinstallation Tasks
6
Denodo Platform 6.0
Installation Guide
PATH=path1;”C:\Program Files\Software” If PATH has double quotes, remove them. 2. PATH cannot end with a backslash (“\”). If it does, the modules of the Denodo Platform will not start as a Windows service. If it does, remove the backslash or add a semicolon at the end of the value of the variable. 2.4.3
Select a User Account to Install the Denodo Platform
In the host where you are installing the Denodo Platform servers, create a user account to install and run the Denodo Platform servers. Important: always install and run the Denodo Platform servers with the same user account. The reason for this is that the Denodo servers modify files in the directory where they are installed and the user account needs read and write privileges. If you perform the installation with one user account and run the servers with another, the second user account may not be able to modify the files in the installation. You can also use an existing account instead of creating one; but you must use the same for both installing and running the Denodo Platform servers. 2.4.4
Upgrading from a Previous Version
If you are upgrading from a previous version, before doing the installation of the new version, read the section “Before Installing a new Version of the Denodo Platform” of the Migration Guide. Once you finish the installation, continue with the rest of the procedures explained in the Migration Guide. 2.5
DOWNLOAD AN INSTALLER
Download the Denodo Platform installer from the Denodo Support Site. Denodo provides several flavors of the installer and the only difference between them is the Java Runtime Environment (JRE) they include. We recommend downloading the installer that includes the 64-bit JRE because it allows assigning more memory to the Denodo servers than the 32-bit JRE.
denodo-install-6.0-win64.zip: it includes a 64-bit JRE for Windows.
denodo-install-6.0-linux64.zip: it includes a 64-bit JRE for Linux.
denodo-install-6.0-linux32.zip: it includes a 32-bit JRE for Linux.
denodo-install-6.0-win32.zip: it includes a 32-bit JRE for Windows.
denodo-install-6.0.zip: it does not include any JRE. Only use this installer if the previous installers are unsuited for your environment. For example, if you are installing on Solaris. Before launching this installer, set the environment variable JAVA_HOME to point to a JRE version 6 or 7. Otherwise, the installer process will fail.
Preinstallation Tasks
7
Denodo Platform 6.0
Installation Guide
In addition to the regular installer, the Denodo Platform also provides the “client installer”. It is directed at developers because it only includes the Virtual DataPort administration tool and the Denodo JDBC driver. Their name is “denodo-install-vdpclient-6.0” and it can also be downloaded from the Denodo Support Site. The installation process is the same as with the regular installer with the difference that it can only install the components mentioned earlier.
Preinstallation Tasks
8
Denodo Platform 6.0
3
Installation Guide
USING THE GRAPHICAL INSTALLATION WIZARD
This section explains how to use the graphical installer of the Denodo Platform. To install the Denodo Platform in a host without graphical environment, see section 4. To do an unattended installation, see the section 8.7. After downloading an installer (see the section 2.5 to see which installer you need), decompress it and launch it by doing the following: On Windows, execute install.exe in the denodo-install-6.0 directory. Launch the installer with administrator privileges. To do this, right-click install.exe and click Run as administrator. If the Windows User Account Control is enabled, you will see a dialog like the following (see Figure 2) in which you have the credentials of an administrator user.
Figure 2
Administrator credentials request dialog
On Linux, execute the following: cd denodo-install-6.0 chmod +x install.sh ./install.sh
The wizard will guide you through the installation steps. In addition, you can always click the Help button of the wizard to obtain assistance during the installation process.
Using the Graphical Installation Wizard
9
Denodo Platform 6.0
Figure 3
Installation Guide
Initial installation screen for the Denodo Platform
Click Next to jump to Step 1. Step 1 The wizard will display the product licensing conditions. Read them carefully and if you accept them, select I accept the terms of this license and click Next. Step 2 Choose between two installation methods: 1. Default Installation: this is the recommended option for most users. You can select the modules you want to install and they will be installed with their default configuration values. With the default installation, the Denodo servers will listen for incoming connections on the default ports. See the list ports in the appendix 11.1. 2. Custom Installation: recommended for advanced users. You will be able to set the values of several configuration parameters such as the ports where the Denodo Platform servers will listen for incoming connections.
Using the Graphical Installation Wizard
10
Denodo Platform 6.0
Figure 4
Installation Guide
Step 2: selecting the installation method
Enter the Installation path where the Denodo Platform will be installed (hereinafter ).
Important: when installing on Windows, we strongly recommend to install the Denodo Platform on the root of the drive (e.g. C:\Denodo\Denodo Platform) and not in “C:\Program files” nor “C:\Program files (x86)”. As the Denodo Platform writes data in the directory where is installed, installing outside “Program Files” avoids issues caused by the protections of the Windows User Account Control (UAC). Important: If you are installing the Denodo Platform on a Windows operating system, the installation path cannot include “junctions” [JUN]. Important: close all the Microsoft Internet Explorer instances before installing the Denodo Platform. Some ITPilot modules need the browser to be closed in order to be installed correctly. IMPORTANT NOTE: if you install the Denodo Platform in a path where there is a previous installation, the existing metadata will be overwritten. Enter the License file location if you have a license provided by Denodo. If you do not have one, leave this box empty. In this case, the Denodo Platform will be installed without a license. You can provide the license after the installation process is complete. The section 5.1 explains how you can do this.
Using the Graphical Installation Wizard
11
Denodo Platform 6.0
Installation Guide
Step 3 Select the modules and components you want to install (see Figure 5).
Figure 5
Step 3: selecting the modules to install
Then, if you select the Custom Installation, you will have to configure each module. The sections 3.1, 3.3, 3.4 and 3.5 explain how to configure the Virtual DataPort, ITPilot, Aracne and Scheduler components, respectively. Before the installer begins copying the necessary files (this process may take several minutes), the installer will display a summary of the installation configuration (see Figure 6). You can save the summary in plain text or HTML by clicking the Save button. Finally, the installation process can create shortcuts in the menus of the operating system to launch the Denodo Platform Control Center (see Section 8.1).
Using the Graphical Installation Wizard
12
Denodo Platform 6.0
Figure 6
3.1
Installation Guide
Summary screen
COMMONS SETTINGS
When you select Custom Installation in the step 2 of the installation wizard, you can configure the internationalization configuration for the Denodo Platform (see Figure 7)
Using the Graphical Installation Wizard
13
Denodo Platform 6.0
Figure 7
3.2
Installation Guide
Choosing the internationalization configuration
COMPONENTS AND CONFIGURATION OF VIRTUAL DATAPORT
You can install the following Virtual DataPort modules:
Administration Tool. The graphical administration tool of Virtual DataPort as well as the classes required to develop applications that query Virtual DataPort.
Information Self Service Tool. Web tool that lets both technical and business users query, search and browse information and metainformation stored in the Virtual DataPort server.
Virtual DataPort Server. The Server that stores the metadata of objects such as data sources, views, etc., and embeds a Web container and execute the queries.
The installer of the Denodo Platform does not include the Diagnostic & Monitoring Tool. This tool was included in the update “20160429” of the Denodo Platform. It is automatically installed when installing this update or newer if you have the Virtual DataPort server installed. This web tool monitors the current state of one or more Virtual DataPort servers and analyzes its state in the past in order to identify the cause of a problem. When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the Virtual DataPort server: Using the Graphical Installation Wizard
14
Denodo Platform 6.0
Installation Guide
Server port number: port that the Virtual DataPort server will listen for incoming connections.
Shutdown port number: port that the Virtual DataPort server will listen for shutdown requests.
Auxiliary port number: auxiliary port used by the Virtual DataPort server to communicate with its clients.
ODBC port number: port that the Virtual DataPort server will listen for incoming ODBC requests.
In Windows operating systems, select Install as a Windows service to install the Virtual DataPort server as a Windows service.
NOTE: If a firewall software is used to control the traffic between the clients and the server, it must be configured to allow communication using these ports. 3.3
COMPONENTS AND CONFIGURATION OF ITPILOT
IMPORTANT NOTE: Before installing ITPilot, close all the instances of Microsoft Internet Explorer. You can install the following ITPilot modules:
Navigation Sequence Generator. Use it to graphically generate automated Web browsing sequences on Web sources. It is installed as a toolbar in the Microsoft Internet Explorer browser. See more about this in the ITPilot Generation Environment Guide [GENITP].
Wrapper Generator Tool. Used to create wrappers on Web sources. See the ITPilot Generation Environment Guide [GENITP].Browser Pool. Wrapper execution environment component required to run automated browsing sequences based on Microsoft Internet Explorer, and that can also use the Denodo browser. See the ITPilot User Guide [ITPILOT].
Wrapper Server. Used to run the wrappers created with the previous components. See the ITPilot User Guide [ITPILOT].
Verification Server. Used to automatically verify that ITPilot wrappers continue running properly after there were changes in the Web sources. See the ITPilot User Guide [ITPILOT].
Administration Tool. Web administration console for executing wrappers and configuring the Browser Pool, the wrapper server and the Verification Server. See the ITPilot User Guide [ITPILOT].
Wrapper Client Environment. API for developing applications that use ITPilot wrappers. See ITPilot Developer Guide [DEVITP].
If you selected Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the ITPilot modules:
Browser Pool: see section 3.3.1.
Using the Graphical Installation Wizard
15
Denodo Platform 6.0
Wrapper Server: see section 3.3.2.
Verification Server: see section 3.3.3.
3.3.1
Installation Guide
Initial Configuration of the Browser Pool
When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the ITPilot Browser Pool:
Server port number: port that the Browser Pool will listen for incoming connections.
Shutdown port number: port that the Browser Pool will listen for shutdown requests.
Auxiliary port number: auxiliary port used by the Browser Pool to communicate with its clients.
Initial browser port: port used by ITPilot to communicate with the first opened Internet Explorer browser. Consecutive ascending port numbers will be used when additional browsers are requested.
In Windows operating systems, select Install as a Windows service to install the Browser Pool as a Windows service.
In the next wizard, configure the path to several application that may be required by ITPilot:
Acrobat Professional installation directory: path where Acrobat Professional is installed. You need this if you are going to extract data from PDF documents using the Adobe Professional software.
Open Office installation directory: path where OpenOffice is installed. You need this if you are going to extract data from Microsoft Word or Microsoft Excel documents.
NOTE: If there is a firewall between the clients and the server, it has to allow opening connections to these ports. 3.3.2
Wrapper Server
When you select Custom Installation in the step 2 of the installation wizard and you do not install the Virtual DataPort server, you can configure the following settings for the ITPilot Wrapper Server:
Server port number: port that the ITPilot Wrapper server will listen for incoming requests.
Shutdown port number: port that the ITPilot Wrapper server will listen for shutdown requests.
Auxiliary port Number: auxiliary port used by the ITPilot Wrapper server to communicate with its clients.
Using the Graphical Installation Wizard
16
Denodo Platform 6.0
Installation Guide
In Windows operating systems, select Install as a Windows service to install the ITPilot Wrapper server as a Windows service.
NOTE: If there is a firewall between the clients and the server, it has to allow opening connections to these ports. If you install the Browser Pool separately from the Wrapper Server, configure the following settings in the Wrapper Server:
Browser Pool IP address.
Browser Pool listening port.
In the event of also installing a browser pool, the values indicated for the pool during installation will be used as values for these parameters. 3.3.3
Verification Server
When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the ITPilot Verification server:
Server port number: port that the ITPilot Verification server will listen for incoming requests.
Shutdown port number: port that the ITPilot Verification server will listen for shutdown requests.
Auxiliary port number: auxiliary port used by the Verification server to communicate with its clients.
In Windows operating systems, select Install as a Windows service to install the ITPilot Verification server as a Windows service.
Furthermore, if a wrapper server is not being installed in the same installation, the wrapper server connection data must be configured:
Wrapper server IP address.
Wrapper server listening port.
In the event of also installing a wrapper server, the values indicated for this server during installation will be used as values for these parameters. NOTE: If there is a firewall between the clients and the server, it has to allow opening connections to these ports. 3.4
ARACNE
You can install the following Aracne modules:
Administration Tool. Web administration console that allows configuring the crawling, filtering, and data indexing tasks. It also allows for queries on the collected data after running the crawling/indexing tasks.
Aracne Server. This includes the crawling modules (except the IECrawler).
Using the Graphical Installation Wizard
17
Denodo Platform 6.0
Installation Guide
IECrawler. Advanced Web crawler based on Microsoft Internet Explorer.
Indexing/Search Engine Server. This includes the indexing and data search servers. If required, these servers can be installed in a different machine to that of the Aracne server.
If you selected the Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the Aracne modules:
Aracne Server: see section 3.4.1.
Indexing/Search Engine Server: see section 3.4.2.
3.4.1
Aracne Server
When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the Aracne server:
Server port number: port that the Aracne server will use to listen for requests.
Shutdown port number: port that the Aracne server will listen for shutdown requests.
Auxiliary port number: auxiliary port used by the Aracne server to communicate with its clients.
In Windows operating systems, select Install as a Windows service to install the Aracne Server as a Windows service.
NOTE: If there is a firewall between the clients and the server, it has to allow opening connections to these ports. 3.4.2
Indexing/Search Engine Server
When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the Indexing/Search Engine server:
Server port number: port that the Indexing/Search Engine server will listen for incoming connections.
Shutdown port number: port that the Indexing/Search Engine server will listen for shutdown requests.
Auxiliary port number: auxiliary port used by the Indexing/Search Engine Server to communicate with its clients.
In Windows operating systems, select Install as a Windows service to install the Indexing/Search Engine Server as a Windows service.
NOTE: If there is a firewall between the clients and the server, it has to allow opening connections to these ports.
Using the Graphical Installation Wizard
18
Denodo Platform 6.0
3.5
Installation Guide
SCHEDULER
You can install the following Scheduler modules:
Scheduler Server. The server for scheduling and configuring batch tasks.
Administration Tool. Web administration tool that allows configuring, scheduling and executing batch data extraction tasks.
When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the Scheduler server:
Server port number: port that the Scheduler server will listen for incoming requests.
Shutdown port number: shutdown requests.
Auxiliary Port number: auxiliary port used by the Scheduler server to communicate with its clients.
In Windows operating systems, select Install as a Windows service to install the Scheduler server as a Windows service.
port that the Scheduler server will listen for
NOTE: If there is a firewall between the clients and the server, it has to allow opening connections to these ports. 3.6
EMBEDDED WEB CONTAINER
The Denodo Platform embeds the Apache Tomcat Web container that is used, among other things, to deploy some of the Denodo Platform administration tools, the Information Self Service Tool, etc. When you select Custom Installation in the step 2 of the installation wizard, you can configure the following settings of the Web container:
HTTP port number: port that the Web container will listen for incoming requests.
Shutdown port number: port that the Web container will listen for shutdown requests.
JMX management port number: port that the Web container will listen for requests from JMX monitoring tools such a Java VisualVM or JConsole. In addition, it is used by the other modules of the Denodo Platform to send requests to the web container such as deploy or undeploy a Denodo administration tool, a web service, etc.
Auxiliary management port number: auxiliary port used by the web container to communicate with its clients. I.e. JMX monitoring tools and other modules of the Denodo Platform.
Using the Graphical Installation Wizard
19
Denodo Platform 6.0
4
Installation Guide
USING THE COMMAND LINE INSTALLER
The command-line wizard is equivalent to the graphical one but it runs on the command line and does not require support for a graphical interface. The steps are the same as in the graphical installer. For example:
There are two types of installation: o
“express” (also known as “Default installation”): this is the recommended option for most users. You can select the modules you want to install and they will be installed with their default configuration values. With the default installation, the Denodo servers will listen for incoming connections on the default ports. See the list ports in the appendix 11.1.
o
“custom” : recommended for advanced users. You will be able to set the values of several configuration parameters such as the ports where the Denodo Platform servers will listen for incoming connections.
You select which modules you want to install.
To start the command-line installer do the following: 1. Decompress the Denodo Platform installer. The section 2.5 explains which one you should download. 2. Execute the following: a. If on Windows, open a command line as an administrator and execute this: cd denodo-install-6.0 installer_cli.bat install b. If on Linux: cd denodo-install-6.0 chmod +x installer_cli.sh ./installer_cli.sh install 3. Follow the steps. The meaning of each parameter is explained in the section that describes how to use the graphical installer (section 3) There are options where at the end you will see something like “(default: 9097)”. This means that if you press Enter, that will be the selected value. Important: when installing on Windows, we strongly recommend to install the Denodo Platform on the root of the drive (e.g. C:\Denodo\Denodo Platform) and not in “C:\Program files” nor “C:\Program files (x86)”.
Using the Command Line Installer
20
Denodo Platform 6.0
Installation Guide
As the Denodo Platform writes data in the directory where is installed, installing outside “Program Files” avoids issues caused by the protections of the Windows User Account Control (UAC).
Using the Command Line Installer
21
Denodo Platform 6.0
5
Installation Guide
POST-INSTALLATION TASKS
After the installer finishes its execution, there are still several tasks required before the software can be run. First, the Platform license file must be installed if it was not provided to the installation wizard. Section 5.1 describes how this is done. Section 5.3 describes the steps to enable SSL for the communications between the Platform modules. If this is not required, that section can be skipped. Subsequently, Sections 5.5, 5.6, and 5.7 explain the post-installation steps for the Virtual DataPort, ITPilot, and Aracne and Scheduler modules, respectively. NOTE: some computer security suites in their default configurations can make the Denodo Platform function incorrectly. If after the installation you cannot create new browsers from the ITPilot Wrapper Generation tool, create browser instances at runtime, or use the MSIE ITPilot toolbar, check that your security software is not blocking any Denodo application or any port used by them. 5.1
INSTALL THE LICENSE
If you did not provide a license file during the installation process or you want to use a different license than the one selected during the installation process, do one of the following: 1. Install the license file using the Denodo Platform Control Center (see section 8.2) 2. Alternatively, rename the desired license file as denodo.lic and copy it to /conf. Without a valid license file, the Denodo Platform applications will not start. 5.2
INSTALL THE LATEST UPDATE
Install the latest update : 1. Go to https://support.denodo.com and download the latest update. 2. Install the update. The section 8.4 explains how to do it. If you just downloaded Denodo Express, you do not need to do this because it already includes the latest update. 5.3
ENABLE SSL CONNECTIONS IN THE DENODO PLATFORM SERVERS
This section explains how to secure with SSL the connections between the Denodo Platform servers, their administration tools and their clients. If you do not need this, jump to the section 5.5. To know how to establish SSL connections with data sources, go to section 5.5.11. Note that if you enable SSL in the Denodo Platform servers, you also have to do it in their clients.
Post-installation Tasks
22
Denodo Platform 6.0
Installation Guide
SSL requires configuring certificate repositories. There are two types of certificate repositories:
KeyStore
TrustStore
KeyStore An application that listens to incoming SSL connections needs a public key and a private key in order to allow clients to access the server. In Java, these keys are stored in a repository called KeyStore. TrustStore During the initialization of an SSL connection, the server sends its SSL certificate to the client. The client must then decide if it trusts this or not. To do this, the client checks if the certificate has been signed by a trusted certification authority (CA). The TrustStore is a repository of the certificates of trusted certification authorities. Every Java installation comes with a TrustStore that the JRE uses by default (/jre/lib/security/cacerts file). If the server’s certificate is not signed by a trusted authority (i.e. one that is not registered in the Java’s TrustStore), you have to store the certificate of the authority, which can be stored in: 1. The cacerts file of the JRE used to launch the Denodo Platform servers and their tools (/jre/lib/security/cacerts file). This is the recommended option because adding the certificate of the authority to this file will make the configuration of the Denodo Platform servers easier. 2. Or, in a new TrustStore. Oracle’s Java Development Kit (JDK) ships with a utility called keytool that manages the Certificate Repositories. 5.3.1
Obtaining and Installing an SSL Certificate
Before enabling SSL in the Denodo Platform servers and clients, you have to create a KeyStore with it. To do this, follow one of these options: 1. If you already have a keystore (a .jks file), go to the next section. 2. If you obtained a file with the SSL certificate in the X.509 format, use keytool (provided by the Java Runtime Environment) to create a KeyStore with it. To do this, execute the following from the command line: cd \jre\bin # If the certification authority (CA) provides an intermediate certificate, execute this command to add it to the keystore and the truststore. The first command below imports the intermediate certificate into the keystore and the second one into the truststore (the cacerts file). keytool -import -trustcacerts -alias Intermediate -keystore ..\..\denodo_server_key_store.jks -file intermediate.cer
Post-installation Tasks
23
Denodo Platform 6.0
Installation Guide
keytool -import -trustcacerts -alias Intermediate -keystore ..\lib\security\cacerts -file intermediate.cer -storepass changeit # Creating the keystore from the certificate. You will have to provide a password. This is the password of the keystore and you will need it later. # When asked to provide the password of the key, press Enter so it has the same password as the KeyStore. keytool -import -trustcacerts -alias denodo-server -keystore ..\..\denodo_server_key_store.jks -file 3. If you do not have an SSL certificate, generate a self-signed one using keytool. To do this, execute the following from the command line: cd \jre\bin # You will have to provide a password. This is the password of the keystore and you will need it later. # When asked to provide the password of the key, press Enter so it has the same password as the KeyStore. keytool -genkeypair -alias denodo-server-self-signed -keyalg RSA keysize 2048 -keystore ..\..\denodo_server_key_store.jks -validity 365 -dname "CN=Denodo, OU=Data Virtualization, O=Acme, L=Palo Alto, ST=California, C=US" # Exporting the public key of the self-signed certificate to a file. You will have to enter the password entered above. keytool -exportcert -alias denodo-server-self-signed -keystore ..\..\denodo_server_key_store.jks -file ..\..\denodo_server_public_key.cer # Importing the public key of the self-signed certificate into the TrustStore of the Denodo Platform JRE. keytool -importcert -alias denodo-server-self-signed -file ..\..\denodo_server_public_key.cer -keystore ..\lib\security\cacerts -storepass changeit
Make sure that the keystore contains the appropriate certificate. To do it, execute this from the command line: keytool -list -v -keystore ..\..\denodo_server_key_store.jks > output_filename Then, open output_filename. You should see something like the following: Alias name: denodo-server... Creation date: Jul 1, 2016 Entry type: PrivateKeyEntry Certificate chain length: ...
Post-installation Tasks
24
Denodo Platform 6.0
Installation Guide
The value of “Entry Type” has to be PrivateKeyEntry or KeyEntry. See the documentation of keytool [JRE-CERT] for a full explanation of how to use this tool. 5.3.2
Enabling SSL in Denodo Platform Servers
Follow these steps to secure with SSL the incoming connections with the servers of a Denodo Platform installation. By doing this, the communications between the Denodo servers and its administration tools, and between the Denodo servers and its clients (JDBC and ODBC applications) will be encrypted. 1. Open the configuration files of the servers whose connections have to be secured: o
Aracne server: /conf/arn/ConfigurationParameters.properties
o
Aracne Index server: /conf/arnindex/ConfigurationParameters.properties
o
ITPilot Browser Pool: /conf/iebrowser/IEBrowserConfiguration.propertie s
o
ITPilot Verification server: /conf/maintenance/ConfigurationParameters.proper ties
o
Scheduler server: /conf/scheduler/ConfigurationParameters.properti es
o
Virtual DataPort server: /conf/vdp/VDBConfiguration.properties
o
Embedded web container (Apache Tomcat): /resources/apache-tomcat/conf/tomcat.properties
2. In all the files opened in the previous step, uncomment the following properties and change their values: o
com.denodo.security.ssl.enabled=true
o
com.denodo.security.ssl.keyStore=Path to the KeyStore that contains the certificate of the Denodo Platform servers. E.g. c:/denodo/denodo_server_key_store.jks Even if the Denodo servers run on Windows, the path separator has to be the forward slash (/).
Post-installation Tasks
25
Denodo Platform 6.0
o
Installation Guide
com.denodo.security.ssl.keyStorePassword=Password of KeyStore containing the certificate of the Denodo Platform servers.
the
If the certificate is not signed by a trusted authority (i.e. one that is not registered in the JRE’s TrustStore), you have to store the certificate of the authority that signed the certificate. If, instead of importing it into the /jre/lib/security/cacerts TrustStore, you have created a new TrustStore, uncomment the property com.denodo.security.ssl.trustStore and set it to the path of the new TrustStore. All the Denodo servers might need the TrustStore because some of them act both as clients and as servers. E.g., Virtual DataPort server receives connections from its Administration Tool, but also establishes connections with the Aracne server. o
Although you can configure the Denodo servers to use a TrustStore that is not the default one (the default TrustStore is at /jre/lib/security/cacerts), we do not recommend it. The reason is that it makes the management of the Denodo servers harder because you have to maintain a new TrustStore file. To use a different TrustStore, uncomment these properties:
com.denodo.security.ssl.trustStore=Path
to
the
TrustStore.
For example, com.denodo.security.ssl.trustStore=/jre/lib/s ecurity/cacerts Even if the Denodo servers run on Windows, the path separator has to be the forward slash (/).
com.denodo.security.ssl.trustStorePassword=Password TrustStore. The default password of the TrustStore (/jre/lib/security/cacerts) is changeit.
of
the
3. To apply these changes, stop all the Denodo Platform servers and once they are all stopped, start them again. It is important to stop them all so the Denodo web container is stopped as well. If for example, you leave the Information Self-Service started the web container will not shut down and the changes in the file tomcat.properties will not take effect. Note: the changes on the file tomcat.properties enable SSL in the communication between the web container and the Virtual DataPort server. To enable HTTPs on the web container, read the following section. 5.3.3
Enabling HTTPS in the Embedded Apache Tomcat
The Denodo Platform embeds the Apache Tomcat web container to host its web applications and web services. The communications between clients and the web applications running in the Apache Tomcat embedded in the Denodo Platform can be secured with HTTPS. The applications running in this web container are: Post-installation Tasks
26
Denodo Platform 6.0
Aracne Administration Tool
ITPilot Administration Tool
Scheduler Administration Tool
Web Services published using Virtual DataPort
Information Self Service Tool
Diagnostic & Monitoring Tool
Installation Guide
To enable HTTPS, do the following: 1. Edit the file /resources/apache-tomcat/conf/tomcat.properties, uncomment the following properties and set their value: o
com.denodo.tomcat.https.port=the port connections. Check that this port is free.
listening
to
HTTPS
If you want clients to access the HTTPs interface without having to put the port in the URL, set this to 443 instead of 9443. That way, the user will be able to access the HTTPs interface with a URL like https://denodo-server/denodo-restfulws instead of https://denodo-server:9443/denodo-restfulws. o
com.denodo.security.ssl.keyStore=Path to the KeyStore that contains the certificate for the Denodo Platform servers. For example, com.denodo.security.ssl.keyStore=c:/denodo/denodo_server_key_ store.jks Even if the Denodo servers run on Windows, the path separator has to be the forward slash (/).
o
com.denodo.security.ssl.keyStorePassword=Password of the KeyStore that contains the certificate for the Denodo Platform servers.
o
It is possible to configure the Denodo web container to use a TrustStore that is not the default one (the default TrustStore is at /jre/lib/security/cacerts). However, we do not recommend doing so because it makes the management of the Denodo servers harder because you have to maintain a new TrustStore file. To use a different TrustStore, uncomment these properties:
com.denodo.security.ssl.trustStore=Path
to
the
TrustStore.
For example, com.denodo.security.ssl.trustStore=/jre/lib/s ecurity/cacerts Even if the Denodo servers run on Windows, the path separator has to be the forward slash (/). Post-installation Tasks
27
Denodo Platform 6.0
Installation Guide
com.denodo.security.ssl.trustStorePassword=Password TrustStore. The default password of the TrustStore (/jre/lib/security/cacerts) is changeit.
of
the
If you want to secure with HTTPS the connections established with Tomcat, but do not want to secure the connections between Tomcat and the Denodo Platform servers, comment the property com.denodo.security.ssl.enabled. 2. Edit the file /resources/apache-tomcat/conf/server.xml a. Uncomment the SSL connector. I.e. Search the “Connector” element that starts with
5.3.4
Enabling SSL in Denodo Platform Tools
When SSL is enabled in a Denodo Platform server, all their clients have to trust the public key of the server, including the Denodo administration tools. If the administration tool is running on a different host than th If, instead of importing it into the /jre/lib/security/cacerts TrustStore, you have created a new TrustStore, do the following. Otherwise, jump to the next section. 1. Open the following files:
Post-installation Tasks
28
Denodo Platform 6.0
Installation Guide
o
/conf/itp-admintool/ITPAdminConfiguration.properties (configuration file of the ITPilot Wrapper Generator Tool)
o
/conf/itpilotclient/ConfigurationParameters.properties
o
/conf/vdp-admin/VDBAdminConfiguration.properties (configuration file of the Virtual DataPort Administration Tool)
o
/tools/monitor/denodomonitor/conf/ConfigurationParameters.properties file of the Denodo Monitor Tool)
(configuration
2. In the files opened in the previous step, uncomment the following property and change its value: o
com.denodo.security.ssl.trustStore=path to the new TrustStore.
The scripts of the Denodo Tools do not have a configuration file. To redefine the default TrustStore that they use, you have to define the javax.net.ssl.trustStore Java system property. For example:
For Windows:
SET JAVA_OPTS= -Djavax.net.ssl.trustStore=/jre/lib/security/cacerts
For Unix:
export JAVA_OPTS= -Djavax.net.ssl.trustStore=/jre/lib/security/cacerts
5.3.5
Enabling SSL for External Clients
5.3.5.1
JDBC and other Java Clients
To secure the communication between Denodo servers and their JDBC clients or other Java clients, set the Java system property javax.net.ssl.trustStore to point to the TrustStore that contains the certificate used by the Denodo servers. For example:
For Windows:
SET JAVA_OPTS= -Djavax.net.ssl.trustStore=/jre/lib/security/cacerts
For Unix:
export JAVA_OPTS= -Djavax.net.ssl.trustStore=/jre/lib/security/cacerts
Some applications allow you to set this property without setting an environment variable:
Post-installation Tasks
29
Denodo Platform 6.0
Installation Guide
JConsole:
jconsole -J-Djavax.net.ssl.trustStore=/jre/lib/security/cacerts In JConsole, when SSL is enabled, enter the URL of the Denodo server with the format : instead of Tools & Services. 2. Uncompress the downloaded package in a temporary directory. 3. Create the directory sap-erp-connector inside /extensions/thirdparty/lib/
Post-installation Tasks
33
Denodo Platform 6.0
Installation Guide
4. Copy the following files to the directory /extensions/thirdparty/lib/sap-erp-connector/ o
sapjco3.jar
o
libsapjco3.so
5. To test that the JCo connector is working, execute the following command: cd cd extensions/thirdparty/lib/sap-erp-connector java -jar \jre\bin\java -jar sapjco3.jar -stdout Note that you are launching the Java Runtime Environment installed along with the Denodo Platform. If you are going to launch the Denodo Platform with another JRE, you have to execute this command with that JRE to make sure that at runtime, the SAP JCo connector will work. If the library is properly installed, this command runs without error and provides information about the installed JCo libraries. 5.5.4
Grant Privileges in SAP BW to the Virtual DataPort User Account
Usually, SAP systems are configured to limit the functions a user can invoke. If you are going to retrieve data from SAP BW (i.e. create multidimensional data sources), you have to grant access to the following functions, via the authorization object S_RFC, to the user account used by Virtual DataPort to connect to SAP BW:
RFCPING
RFC_GET_FUNCTION_INTERFACE
DDIF_FIELDINFO_GET
SYSTEM_RESET_RFC_SERVER
Virtual DataPort invokes these BAPIs at introspection time (when opening the data source to list the SAP BW cubes):
BAPI_MDPROVIDER_GET_CUBES
BAPI_MDPROVIDER_GET_VARIABLES
BAPI_MDPROVIDER_GET_MEASURES
BAPI_MDPROVIDER_GET_DIMENSIONS
BAPI_MDPROVIDER_GET_LEVELS
BAPI_MDPROVIDER_GET_PROPERTIES
BAPI_MDPROVIDER_GET_HIERARCHYS
RSOBJS_GET_NODES_X
Post-installation Tasks
34
Denodo Platform 6.0
Installation Guide
When querying views that involve a multidimensional data source with the “SAP BW 3.x (BAPI)” adapter, it invokes these:
BAPI_MDDATASET_CREATE_OBJECT
BAPI_MDDATASET_GET_AXIS_INFO
BAPI_MDDATASET_GET_AXIS_DATA
BAPI_MDDATASET_GET_CELL_DATA
BAPI_MDDATASET_SELECT_DATA
BAPI_MDDATASET_DELETE_OBJECT
BAPI_MDPROVIDER_GET_MEMBERS
When querying views that involve a multidimensional data source with the “SAP BI 7.x (BAPI)” adapter, it invokes these:
RSR_MDX_CREATE_OBJECT
RSR_MDX_GET_AXIS_INFO
RSR_MDX_GET_AXIS_DATA
RSR_MDX_GET_CELL_DATA
BAPI_MDDATASET_SELECT_DATA
BAPI_MDDATASET_DELETE_OBJECT
BAPI_MDPROVIDER_GET_MEMBERS
5.5.5
Grant Privileges on SAP for BAPI Sources
If you are going to retrieve data from SAP (i.e. create BAPI data sources), you have to grant access to the following functions, via the authorization object S_RFC, to the user account used by Virtual DataPort to connect to SAP:
RFCPING
RFC_GET_FUNCTION_INTERFACE
DDIF_FIELDINFO_GET
SYSTEM_RESET_RFC_SERVER (executed after running the BAPI of a base view)
In addition, grant access to the BAPI invoked by each BAPI base view you create. 5.5.6
Installing the Connector for SAP BW and SAP BI (Multidimensional Sources)
In order to retrieve data from SAP BW or SAP BI, you have to install its connector. Post-installation Tasks
35
Denodo Platform 6.0
Installation Guide
To do this, follow these steps: 1. Enable XMLA access in SAP. You can follow the instructions in the following link to know how to configure XMLA: http://help.sap.com/javadocs/NW04/current/bi/docs/connectors/xmla_howt o.html. Note: You must increase the Web service client time out of SAP BW as explained here: http://wiki.scn.sap.com/wiki/display/JSTSG/%28WSR%29Problems-P01 2. Obtain the SAP Business Intelligence Java Software Development Kit (BI Java SDK) distributed with SAP NetWeaver. The SDK can be downloaded from here: http://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/30232e 2e-e644-2a10-74a4-c482e2203093 3. Uncompress the package and copy all its jars, including the jars in its subdirectories, to /extensions/thirdparty/lib/sap-bw-connector/
5.5.7
Installing the Connector for Oracle Essbase
In order to retrieve data from Oracle Essbase, you have to install its connector. The Oracle Essbase API provides two modes of connecting to Essbase servers. The section “Multidimensional Data Sources to Oracle Essbase” of the Virtual DataPort Administration Guide provides more information about these modes. The set of drivers you have to install, depend on the connection mode used: 1. Three-tier APS mode: obtain the ess_japi.jar. 2. Embedded mode: check the Administration Guide of Oracle Essbase to obtain the list of jars that a Java application needs to connect to Essbase in Embedded mode. Make sure that the version of the Essbase Administration Guide matches the version of your Essbase server because the jars required change depending on the release of Essbase you want to connect to. After obtaining the appropriate jars, copy them to the Denodo installation:
Copy the Oracle Essbase version 9 drivers to the directory /lib/extensions/essbase-drivers/9
Copy the Oracle Essbase version 11 drivers to the directory /lib/extensions/essbase-drivers/11
5.5.8
Enable XMLA on Microsoft (Multidimensional Sources)
SQL
Server
Analysis
Services
If you are going to retrieve data from Microsoft SQL Server Analysis Services, you need to enable its XMLA interface (XML for Analysis).
Post-installation Tasks
36
Denodo Platform 6.0
5.5.9
Installation Guide
Installing the JMS Connectors to Create JMS Listeners and Web services with SOAP over JMS
To connect to a JMS server, you have to install its connector. JMS connectors are a set of jars that have to be copied into the Denodo Platform. You have to do this if you are going to do any of these:
Create JMS Listeners: see section “JMS Listeners” of the Administration Guide [VDP]. If you plan to do this, copy the client jars of the JMS server to the directory /extensions/thirdparty/lib.
Or, create Web services with SOAP over JMS: see section “JMS Listeners” of the Administration Guide [VDP]. If you plan to publish Web services with the option SOAP over JMS enabled and deploy them in the embedded Web container of the Denodo Platform, copy the client jars of the JMS server to the directory /resources/apache-tomcat/common/lib. To deploy Denodo Web services with the option SOAP over JMS enabled, in an external application server, copy the JMS client jars into the /WEB-INF/lib directory of the generated war file, before deploying it.
5.5.9.1
JMS Client Jars
This section lists the client jars of the most popular JMS servers. The vendor provides these jars. Apache ActiveMQ 5.6 The client jars are located in :
activemq-all.jar
IBM WebSphere MQ 7.0 The client jars are located in $WEBSPHERE_MQ_HOME/java/lib:
com.ibm.mq.jar
com.ibm.mqjms.jar
com.ibm.mq.jmqi.jar
dhbcore.jar
fscontext.jar
jta.jar
providerutil.jar
IBM MQ 8.0 The client jars are located in $WEBSPHERE_MQ_HOME/java/lib:
com.ibm.mq.jar
Post-installation Tasks
37
Denodo Platform 6.0
com.ibm.mqjms.jar
com.ibm.mq.jmqi.jar
fscontext.jar
providerutil.jar
Installation Guide
Progress SonicMQ 8.0 The client jars are located in $SONIC_MQ/Sonic/MQ8.0/lib:
mfcontext.jar
sonic_Client.jar
sonic_Crypto.jar
sonic_XA.jar
sonic_XMessage.jar
5.5.10 Configuring the VCS Clients Virtual DataPort can use a Version Control System (VCS) to store the metadata of the Virtual DataPort server (data sources, views, etc.). This allows users to do the main tasks involved in version control from the Administration Tool: check out / update and check in / commit of databases and their elements. The supported Version Control Systems are:
GIT: there are no post-installation tasks to use GIT in Denodo.
Microsoft Team Foundation Server (TFS). The section 5.5.10.3 describes how to configure the Denodo Platform to work with TFS.
Subversion. The sections 5.5.10.1 and 5.5.10.2 describe how to configure your Subversion client to work with the Denodo Platform.
To learn how to use the VCS support of Virtual DataPort, read the section “Version Control Systems Integration” of the Virtual DataPort Administration Guide [VDP]. 5.5.10.1
Configure a Subversion Client to Use the VCS Integration
To use Subversion to store the metadata of Virtual DataPort, perform these tasks: 1. Set up a Subversion server. The supported version is 1.7. 2. In the host where the Virtual DataPort server will run, do the following: a. Install a Subversion client. The version of the client has to be 1.7. The recommended client is Apache Subversion 1.7.x [SVN].
Post-installation Tasks
38
Denodo Platform 6.0
Installation Guide
b. Add to the PATH environment variable the directory where the svn executable is located. For Apache Subversion, that is the bin directory. c. Make sure that the “global ignores” list of the Subversion client does not include any of the following patterns:
d*.a, *.o, (in general, *.)
d*.vql
*.properties
*.dependencies
With Apache Subversion, you have to change the value of the property global-ignores in this configuration file:
On Windows: %APPDATA%\Subversion\config
On Linux: ~/.subversion/config or /etc/subversion/config
The default configuration of Subversion clients includes several file and directory name patterns that are ignored by Subversion operations. For example, by default Apache Subversion 1.7.5, ignores the files that match any of the following patterns: global-ignores = *.o *.lo *.la *.al .libs *.so *.so.[0-9]* *.a *.pyc *.pyo *.rej *~ #*# .#* .*.swp .DS_Store Note that *.o and *.a are included in the list. This is problematic because of the way Virtual DataPort maps folders to their physical location in the file system when exporting to repository or performing VCS operations. For example, a folder named a will be physically located at //folder.a. As all *.a files and directories are ignored by Subversion by default, VCS operations involving such Virtual DataPort folders will fail. Make sure that the global-ignores list does not include any of the mentioned patterns, as they correspond to the types of files involved in VCS operations. Note: some Subversion servers such as “CollabNet Subversion Edge” cannot handle files whose name contains characters reserved by the file system like \, /, :, *, ?, etc. Therefore, we strongly recommend not using any of these characters in the name of the database or any of its elements if this database will be stored in a Subversion server. 5.5.10.2
Configure the Denodo Platform to Work with Subversion
Ignore this section if you installed the Denodo Platform on Windows. If you installed the Denodo Platform on Linux and are going to use Subversion to store the metadata of Virtual DataPort, follow these steps: 1. Launch a Virtual DataPort Administration Tool and log in as an administrator user (e.g. the default admin user). 2. Perform the steps described in the section “Enabling Uniqueness Detection” of the Administration Guide. Post-installation Tasks
39
Denodo Platform 6.0
Installation Guide
3. If you are going to connect to Subversion using the http or https protocol, follow the steps described in the section “Activating the LS Optimization”. 5.5.10.3
Configure the Denodo Platform to Work with Microsoft TFS
To use Microsoft Team Foundation Server (TFS) to store the metadata of Virtual DataPort, you have to set up a Microsoft Team Foundation Server (TFS). The supported versions are 2010 or higher. The Denodo Platform includes the necessary libraries to connect to a TFS server. The TFS administrator is in charge of creating and managing the collections that will contain team projects with Virtual DataPort metadata and their different development branches, if any. We recommend having at least one branch in each project with Virtual DataPort metadata (as described by this MSDN article). The recommended repository structure will be like this (the nodes in italics are managed by the TFS administrators, the others are managed by Virtual DataPort):
TFS Servers o
DefaultCollection
o
Collection 1
o
Collection 2
o
...
o
VDP Collection 1
Team Project 1
Team Project 2
…
Team Project n
Branch-1
Branch-2
…
Branch-n
Main o
Post-installation Tasks
databases
my database 1
my database 2
…
40
Denodo Platform 6.0
Installation Guide
my database n ...
o
extensions
o
environments
o
maps
Each TFS collection is backed up by a different database, so it must be managed separately from the others. Each TFS project can contain several Virtual DataPort databases for each development branch, which will share environments and global elements, so we recommended that each TFS project contain Virtual DataPort metadata related to only one application. 5.5.11 Importing the Certificates of Data Sources (SSL Connections) When Virtual DataPort establishes an SSL connection with a data source, the data source presents a certificate. Virtual DataPort relies on the Java Cryptography Architecture (JCA) to check if the certificate is valid. JCA accepts certificates signed by known Certificate Authorities (CA) (see the list of known CAs here [JRE-CERT]). However, if the certificate used by the server is signed by an authority not present in this list, you have to import this certificate into the list of trusted certificates (called TrustStore). To import a certificate into the TrustStore of the Java Runtime Environment (JRE), execute the following commands: cd cd jre/bin keytool -importcert -alias -file .crt -keystore /jre/lib/security/cacerts This command will prompt you for the password of the TrustStore, which by default is “changeit” (without the quotes). Explanation of the parameters:
alias: this parameter is mandatory. The certificate will be stored in the TrustStore identified by this alias. If the TrustStore already contains a certificate with this alias, use another alias.
keystore: path to the TrustStore where the certificate will be stored. /jre/lib/security/cacerts is the path of the TrustStore of the JRE included in the Denodo Platform. If you have uncommented the property com.denodo.security.ssl.trustStore of the file /conf/vdp/VDBConfiguration.properties, the value of this parameter has to be the value of this property, instead of “/jre/lib/security/cacerts”. That is because, if this property is uncommented, Virtual DataPort will use the TrustStore set in this property of the VDBConfiguration.properties file, instead of the JRE one.
Post-installation Tasks
41
Denodo Platform 6.0
Installation Guide
If you are going to launch Virtual DataPort with a JRE not included in the Denodo Platform and the property com.denodo.security.ssl.trustStore is commented, the value of this parameter has to be the path to the cacerts file of this other JRE, which is located in the directory lib/security of the JRE. To check that the certificate has been imported correctly, execute this command: keytool -list -v -alias \jre\lib\security\cacerts
-keystore
After adding a certificate, you have to restart the Virtual DataPort server in order for the changes to take effect. The keytool documentation [KEYTOOL] provides much more details about the parameters of this tool. 5.5.12 Increasing the Maximum Simultaneous Requests The Denodo Platform embeds the Apache Tomcat Web container that Virtual DataPort uses, among other things, to deploy SOAP and REST Web services. If you expect these Web services to receive a high number of concurrent requests, consider increasing the maximum number of threads that Tomcat will create to attend requests. To do this, follow these steps: 1. Open the file /resources/apache-tomcat/conf/server.xml 2. Look for the attribute maxThreads and replace its default value (150) with a higher one. For example, 300. There are two occurrences of the maxThreads attribute, one for the non-SSL Connector and another one for the SSL Connector. You can have a different value for each one. 3. From the Administration Tool, do the following: a. Open the wizard “Concurrent Requests” on the menu “Administration > Server configuration”. b. Make sure that if “Limit concurrent requests” is “On”, the value of “Max concurrent requests” is greater than or equal to the maxThreads attribute. Otherwise, Tomcat will process the requests, but Virtual DataPort will not be able to attend them. If you have enabled SSL on Tomcat (explained in the section 5.3.3 Enabling HTTPS in the Embedded Apache Tomcat), “Max concurrent requests” has to be greater than or equal to the sum of the maxThreads attribute of the non-SSL connector and the SSL connector. 4. To apply these changes, stop the Virtual DataPort server and all the Denodo administration tools. Once they are all stopped, start them again. Each incoming request requires a thread for the duration of that request. If Tomcat receives more simultaneous requests than the number of available request processing threads, Tomcat creates additional threads up to maxThreads. If Tomcat still receives more simultaneous requests, they are stacked up, up to the value of the Post-installation Tasks
42
Denodo Platform 6.0
Installation Guide
acceptCount attribute of the Connector element. Any further simultaneous requests will receive "connection refused" errors. 5.5.13 Setting-up Kerberos Authentication in Virtual DataPort Virtual DataPort provides support to authenticate its clients using the Kerberos authentication protocol. Before enabling this support in Virtual DataPort, you have to perform these tasks: 1. Configure the Kerberos authentication server (e.g. Active Directory) to be able to authenticate the Virtual DataPort server and its users: see section 5.5.13.1. 2. If necessary, install the Java Cryptography Extension (JCE): see section 5.5.13.2. 3. Modify the Windows registry to use the native ticket cache: see section 5.5.13.3 After performing these steps, you have to configure the Virtual DataPort server and its clients to use Kerberos authentication. The section “Kerberos Authentication” of the Virtual DataPort Administration Guide explains how to do this. 5.5.13.1
Configuring the Kerberos Authentication Server (e.g. Active Directory)
You need to perform the following tasks on the Kerberos server, which on a Windows environment is Active Directory: 1. Create a user in the Kerberos server of type “User”: see section 5.5.13.1.1. 2. Declare a Service Principal Name (SPN) and associate it with the user of the Virtual DataPort server: see section 5.5.13.1.2. 3. Generate a keytab file: see section 5.5.13.1.3. 5.5.13.1.1
Creating a User in the Active Directory
Create a Kerberos Principal in Active Directory. The Virtual DataPort server will use this principal to offer Kerberos authentication to its clients. To create a user called denodo_server, follow these steps: 1. Launch Programs/Administrative Tools/Active Directory Users and Computers tool. 2. Right click the Users node and select New/User (do not select Machine). 3. Enter denodo_server in the “Full Name” and the “Logon name” fields. 4. Click Next and enter a password. 5. Clear all the password options and click Next and then Finish. 6. Locate denodo_server in the “Users” tree, in the left side pane and doubleclick it.
Post-installation Tasks
43
Denodo Platform 6.0
Installation Guide
7. Click the “Account” tab of denodo_server and select the cipher strength used in the communications between the Virtual DataPort server and the Active Directory: o
To use AES128, select This account supports AES 128 bit encryption and clear the other encryption check boxes.
o
To use AES256, select This account supports AES 256 bit encryption and clear the other encryption check boxes. If you select this encryption mechanism, you have to install the “Java Cryptography Extension (JCE)” in the host where the Virtual DataPort server runs and in all the clients that connect to Virtual DataPort using Kerberos authentication. Using AES128-SHA does not require installing this extension. The section 5.5.13.2 explains how to install this extension.
o
To use RC4-HMAC-NT, clear all the check boxes except password never expires.
o
For DES, select Use Kerberos DES encryption and clear the other encryption check boxes.
To be able to use “pass-through session credentials” when using Active Directory (see explanation in the previous section), enable the option "Trust this user for delegation to any service (Kerberos only)" for this user (see figure below). Important: if you select Trust this user for delegation to specified services only, you are enabling what it is called “constrained delegation”. In this case, the queries sent to Virtual DataPort could fail if they involve a JDBC data source with the authentication option “pass-through session credentials”. Read the section “Connecting to a JDBC Source with Kerberos Authentication” of the Administration Guide to check if in your scenario you can enable this option.
Post-installation Tasks
44
Denodo Platform 6.0
Figure 9
5.5.13.1.2
Installation Guide
Active Directory User Configuration
Declaring a Service Principal Name (SPN)
An SPN (Service Principal Name) is a unique name that identifies an instance of a service and is associated with the logon account under which the service instance runs. The SPN is used in the process of mutual authentication between the client and the server hosting a particular service. Use the setspn utility to declare an SPN and associate it with the user account that was created in the previous step setspn -a HTTP/ \ Figure 10
Syntax of the setspn utility
is the Fully Qualified Domain Name of the host where the Virtual DataPort server runs. For example,
Post-installation Tasks
45
Denodo Platform 6.0
Installation Guide
setspn -a HTTP/host1.subnet1.contoso.com CONTOSO.COM\denodo_server Figure 11
Example of declaring a Service Principal Name with setspn
See the full documentation us/library/cc731241.aspx.
of
setspn
at
https://technet.microsoft.com/en-
See what a Fully Qualified Domain Name is at https://en.wikipedia.org/wiki/Fully_qualified_domain_name. Note that you have to enter the realm http://support.microsoft.com/kb/248807/en-us. 5.5.13.1.3
in
uppercase
as
explained
in
Generating a Keytab File for Virtual DataPort
Generate a keytab file so the Virtual DataPort server can authenticate itself and its clients with the Active Directory using Kerberos. A keytab file contains pairs of Kerberos principals and encrypted keys derived from the Kerberos password. To generate this file, use the ktpass utility. ktpass -out denodo.keytab -princ @ -mapUser -crypto ALL -pass * -ptype KRB5_NT_PRINCIPAL Figure 12
Syntax of the ktpass utility
See the full documentation of the ktpass utility at https://technet.microsoft.com/en-us/library/cc753771.aspx. For example: ktpass -out denodo.keytab -princ HTTP/
[email protected] -mapuser denodo_server pass MyPassword -crypto ALL -ptype KRB5_NT_PRINCIPAL Figure 13
Example of generating a keytab file
You have to execute this in the host of the Active Directory and then, copy the generated keytab file to the host where the Virtual DataPort server will run. 5.5.13.2
Installing the Java Cryptography Extension (JCE)
If you configured the user account to use the AES256 encryption method, install the Java Cryptography Extension (JCE) in the Virtual DataPort server. To do this, follow these steps: 1. Download the JCE: http://www.oracle.com/technetwork/java/javase/downloads/jce-7download-432124.html 2. Make a copy of the original JCE policy files (US_export_policy.jar and local_policy.jar of the folder /jre/lib/security/). This will allow reverting to the original policy versions.
Post-installation Tasks
46
Denodo Platform 6.0
Installation Guide
3. Copy the jars inside the downloaded file to /jre/lib/security Note: if you configured Denodo to run with a JRE other than the one included by Denodo, copy these files to the lib\security folder of JRE you are using, instead of to the Denodo’s JRE. Important: you also have to perform these steps in all the clients of Virtual DataPort that want to use Kerberos authentication. That includes the Virtual DataPort administration tools and the JDBC clients. 5.5.13.3
Modifying the Windows Registry to use the Native Ticket Cache
Windows and Linux provide “Single Sign-on (SSO)” and you can take advantage of this to connect to a Virtual DataPort server that is configured with Kerberos authentication. This will allow users to connect to Virtual DataPort without having to provide a password because they will rely on the operating system’s authentication. You do not have to do anything to use single sign-on on Linux. On Windows, you have to modify the registry so the clients of Virtual DataPort (the Administration Tool or JDBC applications) can use single sign-on. To do this, follow these steps: 1. Run regedit.exe 2. Look for the key HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa\Kerberos \Parameters 3. Right-click the “Parameters” node and click New > DWORD. 4. Enter the name of the entry: allowtgtsessionkey. 5. Double click on the new entry and set the value to 1 (0x00000001) Important: You have to do this in all these hosts:
All the hosts where Virtual DataPort administration tools run and whose users want to use Kerberos authentication.
All the hosts where JDBC clients run and want to use Kerberos authentication
You have to do this when the client runs on Windows because, as explained in the Troubleshooting page for Kerberos authentication of the Java Runtime Environment, by default Windows does not give access to the session key of a Ticket-Granting Ticket (TGT). This change in the registry will make the session key for TGT accessible, so Java can use it to acquire additional service tickets. If you do not do this, you will be able to use Kerberos authentication, but not Single Sign-on. 5.5.14 Setting-up Kerberos Authentication in the Information Self-Service Tool The Information Self-Service Tool provides support to authenticate its clients using the Kerberos authentication protocol. Post-installation Tasks
47
Denodo Platform 6.0
Installation Guide
Once you have set-up each Virtual DataPort server to use Kerberos, it is important to distinguish between these two scenarios: 1. The Information Self-Service Tool server is in the same machine as the Virtual DataPort server: see section 5.5.14.1. 2. The Information Self-Service Tool server is in a different machine than the Virtual DataPort server: see section 5.5.14.2. 5.5.14.1
The Information Self-Service Tool and the Virtual DataPort server are in the same machine
In this case, to configure the Information Self-Service Tool, you can use the same user from the Active Directory and the keytab file generated during the postinstallation tasks of Virtual DataPort described in the section 5.5.13. As an alternative, you can create a new user and generate a new keytab for that user. The only restriction is that in a LDAP server the SPNs are unique. This way, if there is already a user with an SPN HTTP/, you cannot create a new user with that SPN. Note that the SPN for the Information Self-Service Tool has to start with “HTTP/”, which is the protocol that will be used, as the user is going to use a browser. 5.5.14.2
The Information Self-Service Tool and the Virtual DataPort server are in different machines
In this case, you will have to perform similar steps as the ones described in the section 5.5.13 (but applied to the Information Self-Service Tool server instead of the Virtual DataPort server). 1. Configure the Kerberos authentication server (e.g. Active Directory): see section 5.5.14.2.1. 2. Install the Java Cryptography Extension (JCE): see section 5.5.14.2.2. After performing these steps, you have to configure the Information Self-Service Tool to use Kerberos authentication. The section “Kerberos Configuration” of the Information Self-Service Tool Administration Guide explains how to do this. 5.5.14.2.1
Configuring the Kerberos Authentication Server (e.g. Active Directory)
You need to perform the following tasks on the Kerberos server, which on a Windows environment is Active Directory: 1. Create a user in the Kerberos server of type “User”: see section 5.5.13.1.1. 2. Declare a Service Principal Name (SPN) and associate it with the user of the Information Self-Service Tool: see section 5.5.13.1.2. 3. Generate a keytab file (see section 5.5.13.1.3). You have to execute this in the host of the Active Directory and then, copy the generated keytab file to the host where the Information Self-Service Tool will run.
Post-installation Tasks
48
Denodo Platform 6.0
5.5.14.2.2
Installation Guide
Installing the Java Cryptography Extension (JCE)
Explained in 5.5.13.2. In this case, the extension has to be installed only in the machine running the Information Self-Service Tool server because the clients will be web browsers and do not need it. 5.5.15 Enabling the Support for ODBC Sources When the Virtual DataPort Server Runs on Linux Virtual DataPort provides access to ODBC data sources. However, this feature is disabled by default when the Virtual DataPort server runs on Linux. The reason for being disabled by default is that on Linux, depending on the configuration of the host, Virtual DataPort could load the wrong library to connect to ODBC sources. If the wrong library is loaded, Virtual DataPort crashes. To avoid that a user creates an ODBC source that leads the Virtual DataPort server to crash, ODBC sources are disabled by default. Follow these steps to enable this feature: 1. Install the package unixODBC. The section “Install unixODBC” of the Developer Guide explains how to do it. 2. Log into the user account you will use to launch the Virtual DataPort server. 3. Check that these files exist: /usr/local/lib/libodbc.so /usr/local/lib/libodbcinst.so These files are provided by the unixODBC package. Depending on the Linux distribution, they could located at /usr/local/lib64/. 4. Edit the file ~/.bash_profile and add the following at the end. export LD_PRELOAD=/usr/local/lib/libodbc.so:/usr/local/lib/libodbcinst.so:$ LD_PRELOAD With this change in the value of the variable LD_PRELOAD, you make sure that Virtual DataPort loads the files libodbc.so and libodbcinst.so provided by unixODBC and not the ones provided by other libraries. Note: if the two files listed above are in lib64 and not in lib, change the line above accordingly. 5. Logout and login again from this user account. Do this to apply the changes done in .bash_profile. 6. If the Virtual DataPort server was started, stop it. 7. Start the Virtual DataPort server and login with an administrator account. Then, execute this command on the VQL Shell: SET 'com.denodo.vdb.ODBCDataSource.enable'='true';
Post-installation Tasks
49
Denodo Platform 6.0
Installation Guide
This command enables the support for ODBC sources on Linux. 8. To check that the configuration has been updated correctly, do the following from the administration tool: a. Create an ODBC data source. b. Create an ODBC base view. c. Query this base view. 5.5.16 Resource Manager: Modify the Priority of a Query Dynamically The Resource Manager of Virtual DataPort allows, among other things, to modify the priority of the queries executed. See more about feature in the section “Resource Manager” of the Administration Guide. If you are planning to use modify the priority of the queries and the Virtual DataPort server runs on Linux, you have to do the following: 1. Start the Virtual DataPort with the root user. 2. And add this parameter to the “JVM options” of the Virtual DataPort server: -XX:ThreadPriorityPolicy=1. The section 8.5 explains how to modify the JVM options of each module. The reason for having to do this is that on Linux, only processes launched by the root user can change the priority of its threads dynamically. If these conditions are not met, the priorities of the queries will not change even if a restriction plan indicates so. This reminder does not affect Virtual DataPort servers that run on Windows; on Windows, processes can change the priority of its threads. 5.5.17 Launching the Administration Tool in High DPI Displays If you are going to launch the Virtual DataPort administration tool in a computer with a high DPI display, follow the steps of the appendix 11.7. 5.6
POST-INSTALLATION TASKS IN ITPILOT
After installing ITPilot, perform the tasks described in the following subsections. 5.6.1
Checking that the Generation Environment Has Been Installed
The module ITPilot Generation Environment installs a toolbar in the Internet Explorer browser that is used to aid in the wrapper generation process (recording browsing sequences and generating data extraction programs). Follow these steps to check that the toolbar was installed correctly: 1. Launch Microsoft Internet Explorer. 2. The navigation sequences generator toolbar should be visible on the browser. Post-installation Tasks
50
Denodo Platform 6.0
Installation Guide
If it is not, activate it by clicking Sequence Generator in the View > Toolbars menu. See the section 5.6.3 to enable the toolbar in Microsoft Windows 2008. NOTE: If several versions of the platform have been installed in the same machine, only the Sequences Generator Toolbar of one version should be enabled in Internet Explorer at the same time. 5.6.2
Configuration of Microsoft Internet Explorer
To run automated browser sequences, ITPilot uses a Browser Pool that will spawn browsers based on Microsoft Internet Explorer or Denodo Browser. To be able to use Internet Explorer, you have to configure it appropriately. It is very important that you perform these steps using the same user account that you will use to launch the Denodo Control Center and under which the Denodo Windows services will run. 1. Open Internet Explorer and configure it with the settings required by your environment: proxy options, security level, cookies, etc. The reason is that the browsers opened by the pool will use this configuration. 2. Make sure that “Active Scripting” is enabled (this option is disabled by default in some Windows Server versions). To do this, follow these steps: a. On the menu Tools of Internet Explorer, click Internet options. b. Click the tab Security. c. Click Custom level. d. Look for the category Active scripting, inside the category Scripting, and click Enable. 3. If using Internet Explorer 10 and 11, follow these steps: o
On 32-bit Windows, follow these steps: i.
Execute regedit.exe.
ii. Browse to HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Internet Explorer > Main. iii. Add a DWORD value called TabProcGrowth with value 0. o
On 64-bit Windows, follow these steps: i.
Execute the 64-bit version of regedit.exe (this is the version included in the PATH by default).
ii. Browse to HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Internet Explorer > Main. iii. Add a DWORD value called TabProcGrowth with value 0.
Post-installation Tasks
51
Denodo Platform 6.0
Installation Guide
iv. Browse to HKEY_LOCAL_MACHINE > SOFTWARE -> Wow6432Node > Microsoft > Internet Explorer > Main. v. Add a DWORD value called TabProcGrowth with value 0. You have to do this because Internet Explorer, in the versions 10 and 11, spawns each tab as a separate process of the operating system. To record navigation sequences (NSEQL sequences), ITPilot requires Internet Explorer to use the same process for all tabs. 5.6.3
Enabling Internet Explorer Sequence Microsoft Windows Server 2008
Generation
Toolbar
in
To show the Internet Explorer toolbars in a Microsoft Windows Server 2008 operating system, you have to enable third-party browser extensions. To do this, follow the steps below. It is very important that you perform these steps using the same account that you will use to launch the Denodo Control Center and the Denodo Windows services. 1. Close all instances of Internet Explorer, click Start, open Settings, and then click Control Panel. 2. Double-click Internet Options. 3. Click the Advanced tab. 4. Under the category Browsing, select the Enable third-party browser extensions (requires restart). 5. Restart Internet Explorer. The previous steps are equivalent to set value "Enable Browser Extensions"="yes" in the registry key "HKCU\Software\Microsoft\Internet Explorer\Main". 5.6.4
Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows 2008
In Microsoft Windows Server 2008, the Internet Explorer Enhanced Security Configuration feature interferes with the correct functioning of the Sequence Generation Toolbar, and has to be disabled. Follow these steps to disable this feature: 1. Log on to the computer with a user account that is a member of the local Administrators group. 2. Click Start, go to Administrative Tools, and then click Server Manager. 3. If the “User Account Control” dialog box appears, click Continue. 4. Under Security Summary, click Configure IE ESC. 5. Set IE ESC to Off for the appropriate user type (Administrators, Users).
Post-installation Tasks
52
Denodo Platform 6.0
Installation Guide
6. Click OK. 7. Restart Internet Explorer. Do this in all the hosts where a developer is going to use the “Sequence Generation Toolbar” to develop ITPilot wrappers. 5.6.5
Disabling Internet Explorer Enhanced Security Configuration in Microsoft Windows Server 2012
In Microsoft Windows Server 2012, the Internet Explorer Enhanced Security Configuration feature interferes with the correct functioning of the Sequence Generation Toolbar, and has to be disabled. Follow these steps to disable this feature: 1. Log on to the computer with a user account that is a member of the local Administrators group. 2. Click Start, go to Administrative Tools, and then click Server Manager. 3. If the “User Account Control” dialog box appears, click Continue. 4. Click Local Server. 5. On the right side of the “Server Manager”, look for the “IE Enhanced Security Configuration Setting” (its default value is “On”). Set it to Off for the appropriate user type (Administrators, Users). 6. Click OK. 7. Restart Internet Explorer. Do this in all the hosts where a developer is going to use the “Sequence Generation Toolbar” to develop ITPilot wrappers. 5.6.6
Manual Configuration of Adobe Acrobat Professional
During the installation process, you can enter the path to the plugins directory of Adobe Acrobat Professional. If you did not do it, you can configure it later by doing one of these actions:
By setting it in the ITPilot Administration Tool or in the Browser Pool tabs (see [ITPILOT] for more information).
Or, in the Document Conversion tab of the ITPilot Wrapper Generation Tool’s preferences (see [GENITP] for more information)
5.6.7
Automatic Verification Database
The Verification server uses a relational database to store information about wrappers execution to allow detecting when the sources changes and the wrappers stop working properly. ITPilot provides an embedded Apache Derby [DERBY] database
Post-installation Tasks
53
Denodo Platform 6.0
Installation Guide
that can be used for this purpose. If the embedded database is going to be used, no action is required in this section. An external JDBC database management system can also be used. In the current ITPilot version, the supported databases are MySQL [MYSQL] and Oracle [ORCL]. ITPilot provides a script to create the table for these Database Management Systems in the path /scripts/itpilot/sql. If an external database is going to be used for this purpose, it is needed to install the database and run on it the corresponding tables creation script. 5.6.8
Launching the Wrapper Generator Tool in High DPI Displays
If you are going to launch the ITPilot Wrapper Generator Tool in a computer with a high DPI display, follow the steps of the appendix 11.7. 5.7
POST-INSTALLATION TASKS IN ARACNE AND SCHEDULER
After installing Aracne and Scheduler, perform the tasks described in the following subsections. 5.7.1
Microsoft Internet Explorer Configuration
When using the IECrawler module of Scheduler and Aracne, its browsers will use the configuration settings of Microsoft Internet Explorer (MSIE):
We recommend setting the MSIE home page to a blank page (“about:blank”). Otherwise, each new browser launched by IECrawler will load the home page before running a sequence.
For HTTPS site crawling to work properly, the MSIE preferences must be established to prevent pop-up windows from informing users of the change to secure/insecure mode and to accept digital certificates (if not, the browsers may be blocked). To change this, open the dialog “Internet Options”, on the Tools menu, click the “Advanced Options” tab and then, the “Security” option group.
To connect to an FTP server using IECrawler, the Active or Passive connection mode of the MSIE FTP client must be configured. The correct value will depend on the firewall software configuration in the client and in the FTP server. The preferences to establish the MSIE FTP client connection mode are in the “Internet Options” option of the Tools menu, selecting the “Advanced Options” tab and then the “Browse” option group. The MSIE connects in Active mode if the “Enable the list of folders for FTP sites” option is selected, even if the “Use passive FTP” option is selected. If the “Enable the list of folders for FTP sites” option is not selected and the “Use passive FTP” option is selected, the MSIE connects in Passive mode.
Due to the fact that the browsers behave in line with the configuration settings of MSIE, the other MSIE security and cookie options should also be taken into account.
Post-installation Tasks
54
Denodo Platform 6.0
6
Installation Guide
UNINSTALLATION
In the uninstallation process, you can uninstall the entire Denodo Platform or just some of its modules. Before starting the uninstallation process, close all the Denodo servers and tools and all the Microsoft Internet Explorer instances. Otherwise, the folder in which the software was installed will not be deleted. To uninstall, open the Control Center and click Uninstall. The section 8.6 provides more details about this.
Uninstallation
55
Denodo Platform 6.0
7
Installation Guide
DENODO4E
The Denodo Platform includes Denodo4E: an Eclipse plug-in that provides tools for creating, debugging and deploying Denodo extensions. The Denodo4E plug-in has been tested with the following Eclipse versions:
Eclipse Kepler (4.3)
Eclipse Luna (4.4)
To install the plug-in, follow these steps: 1. Click Install New Software on the menu Help. 2. Click the button Add. 3. Click Local... to open the “Add Repository” dialog (see Figure 14). In this dialog, select the file /tools/denodo4e/updateSite and click Ok. Then introduce a name for the new update site. For example "Denodo", and click OK.
Figure 14
Configuring the local repository for Denodo 4E
4. Select Denodo4E (located under the Denodo Technologies category) 5. Select the check box Contact all update sites during install to find required software. 6. Click Next. Once Eclipse finishes calculating dependencies, click Finish to begin the installation process. 7. Once all the required features are installed, restart Eclipse.
Denodo4E
56
Denodo Platform 6.0
8
Installation Guide
DENODO PLATFORM CONTROL CENTER
The Denodo Platform Control Center allows starting and stopping all Denodo Platform servers and tools, as well as a group of additional functions. This section describes its use. 8.1
EXECUTION
To launch the Denodo Platform Control Center do the following:
Use the shortcuts created on the desktop during the installation and/or the application menus of your Operating System
Or, execute /bin/denodo_platform
NOTE 1: In Windows Vista, Windows 7 and Windows Server 2008, we recommend disabling the User Account Control (UAC) before launching the Denodo Platform Control Center. UAC cannot be disabled in Windows 8 or in Windows Server 2012. If you do not want to disable UAC or it cannot be disabled in your operating system, keep in mind the following issues:
Launch the Control Center (\bin\denodo_platform.exe) with the “Run as Administrator” option of the contextual menu of denodo_platform.exe, even if you are already logged in as an administrator user. If you launch a “standalone” Internet Explorer (i.e. it is not launched from the ITPilot Wrapper Generation Tool), you also may need to explicitly use the “Run as Administrator” option to properly use the ITPilot Sequence Generator Toolbar.
If you are using the PDF, Excel or Word conversion features, you may have to disable the Internet Explorer protected mode (this can be done from the Security page of the Internet Options dialog of Internet Explorer).
NOTE 2: When you launch any Denodo server, an icon will be displayed in the notification area of your operating system. This icon will not be displayed when using a 64-bit Java Virtual Machine to launch the server. 8.2
CONTROL CENTER HELP
The first time you launch the Denodo Platform Control Center you will see the Help screen (see Figure 15). If you clear the Show on startup check box, the first thing you will see the next time you launch the Control Center will be the “Custom environments” screen (see section 8.3.1 for details). To return to the Help screen, click Help on the right side of the Control Center.
Denodo Platform Control Center
57
Denodo Platform 6.0
Figure 15
Installation Guide
Denodo Platform Control Center Help screen
The “Help” screen is divided in three sections:
Documentation & Tutorials o
Click Documentation to open the on-line documentation in your web browser.
o
Click Tutorials to open the tutorials in your web browser.
o
Click Online help to configure the embedded help included with the Denodo Platform. The section 8.2.1 for details).
License o
Click License information to display the conditions of your license (see Figure 16).
o
Click Install license to install a new license file. Note that when doing so, you have to restart all the modules and tools for the changes to take effect.
Denodo Platform Control Center
58
Denodo Platform 6.0
Figure 16
8.2.1
Installation Guide
Information on the current license
Online Help Configuration
The location of the help contents can be configured in the online help configuration dialog (see Figure 17). By default, the ITPilot Wrapper Generation Tool and the VDP Administration Tool will show the help located in the local web container when requested. Alternatively, a remote help container can be configured. In this case, the help web applications included with the Denodo Platform should be deployed so they become available at /help/vdp/platform/6.0/vdpadmintool and/or /help/itp/platform/6.0/itpgentool. The corresponding war files can be found at /webapps/help.
Figure 17
Denodo Platform Control Center
Online help configuration dialog
59
Denodo Platform 6.0
8.3
Installation Guide
STARTING PLATFORM SERVERS AND TOOLS
The buttons on the left side of the Control Center (see Figure 18) show the dialog of each module. Each one of these dialogs allows you to start/stop the servers and graphical tools of the module and display their status.
Figure 18
Denodo Platform Control Center
In the dialog of each module, there are two sections: 1. Tools, on the upper part: that is, desktop and web administration tools. 2. Servers, on the lower part of the dialog. To start a server, click the button To stop a running server, click
next to its name. to stop it.
Click Stop all servers in the lower-left side to stop all the currently launched servers and web applications of the Denodo Platform. To launch an administration tool, click Launch next to its name. If it is a web tool, the main page of the tool will be opened on your browser. The URL for the web tools are:
Aracne Administration aracne-admin/
Denodo Platform Control Center
Tool:
http://localhost:9090/webadmin/denodo-
60
Denodo Platform 6.0
Installation Guide
ITPilot Administration Tool: http://localhost:9090/webadmin/denodo-itpilotadmin/
Scheduler Administration scheduler-admin/
Information Self Service Tool: http://localhost:9090/information-self-servicetool
Diagnostic & Monitoring Tool: http://localhost:9090/diagnostic-monitoringtool
Tool:
http://localhost:9090/webadmin/denodo-
The list below refers to the documents that explain how to launch the servers/tools of each module, with a script:
Virtual DataPort: “Launching the Virtual DataPort Server” and “Launching the Virtual DataPort Administration Tool” sections of the Virtual DataPort Administration Guide [VDP].
ITPilot: “Execution” section of the ITPilot User Guide [ITPILOT].
Aracne: “Installation and Execution” section of the Aracne Administration Guide [ARCN]
Scheduler: “Installation and Administration Guide [SCH].
8.3.1
Execution”
section
of
the
Scheduler
Configuration of Custom Environments
In the Control Center you can define “Custom environments”, which are collections of servers and administration tools that can be managed as a group, instead of individually. For example, you can create an environment with all the administration tools you usually use so you can start them all at once instead of starting them one by one. To manage custom environments click Custom on the left side of the Control Center (see Figure 19).
Denodo Platform Control Center
61
Denodo Platform 6.0
Figure 19
Installation Guide
Creation of custom environments
There is a default custom environment: Data Integration Environment that includes the Virtual DataPort server, its administration tool and the ITPilot Browser Pool. To create a new environment, click
and in the wizard (see Figure 20), provide this:
1. Enter the name of the environment. 2. Select the modules of the environment. 3. Select the Java Virtual Machine that will be used to start the modules of this environment.
Denodo Platform Control Center
62
Denodo Platform 6.0
Figure 20
Installation Guide
Creation and editing of custom environments
To edit an environment, click To delete an environment, click
. .
To start all the components of an environment, select it and click of the wizard. To stop all the components of an environment, select it and click 8.4
on the bottom .
INSTALLING UPDATES AND HOTFIXES
The Control Center displays the version the latest installed update or hotfix, next to the name of each product. The updates or hotfixes are released in a zip file, which include the following:
A RELEASENOTES file for each module that will be updated. These files describe the bug fixes and enhancements.
A jar file that contains the update.
Beta updates include a temporary license that allow you install the beta update on a separate environment without interfering with you current deployments.
To install a new update or a hotfix, follow these steps: 1. Close all the Denodo programs and tools that are running on the host where you are installing the update or hotfix. 2. Close all the instances of Microsoft Internet Explorer. 3. Although not mandatory, on the production servers we recommend copying the folder of the Denodo Platform before installing a new update. Denodo Platform Control Center
63
Denodo Platform 6.0
Installation Guide
This copy will allow you to restore quickly the Platform to its previous state, if necessary. 4. Decompress the zip file of the update. 5. Read the RELEASE NOTES file of each product. Pay special attention to the “Post-installation actions” of each product. 6. Open the Control Center. 7. Click Update to open the “Platform Updates” dialog. This dialog lists the updates that have been installed (the last update of the list is the current one). 8. Click Install and select the jar file of the update. The Control Center will display a dialog with a progress bar. The section 9.3 explains how to install an update or a hotfix in a host without graphical support. Note: if in your environment, the Denodo servers run on one host and the Virtual DataPort administration tool on others, you can have a newer update in the server than in the administration tool and JDBC/ODBC clients. That is, the Denodo server is backward compatible regarding its administration tool and its drivers. 8.5
VIRTUAL MACHINE AND WEB CONTAINER CONFIGURATION
Click Configure to open the “Denodo Platform Control Center configuration”. In this dialog, you can do the following:
Select the Java Virtual Machine (JVM) used to run the Denodo Platform servers.
The JVM parameters used to launch each Denodo server.
The ports that the Denodo embedded Web container listens for incoming connections.
To select the Java Virtual Machine (JVM) used to run the Denodo Platform servers, click Configure and then, Edit. In this dialog, you can add a JVM present on your system that you want the Denodo Platform servers to run with. The Denodo servers will run with the JVM that is selected on the list. Although it is possible to change the JVM, we recommend running the Denodo servers with the “Denodo (Internal JVM)”, unless you have used the denodo-install6.0.zip installer, which does not include a Java Runtime Environment (JRE). To configure the JVM options of each Denodo server, click Configure and then, JVM options. This dialog has two tabs: 1. Memory options. In this tab, you can change the JVM parameters of each Denodo server. The section 8.7 explains how to change these parameters in a host without graphical support. Denodo Platform Control Center
64
Denodo Platform 6.0
Installation Guide
2. RMI host. In this tab, you can configure the RMI host name for each server of the Denodo Platform. You have to change this if the host where the Denodo server runs provides several network interfaces. In this case, you have to specify a network interface (as an IP address or a host name) that is visible to clients that have to connect to this server. In both tabs, next to each field there is a button ( ) that can be used to go back to the default value. The Ok button will save the configuration changes, which will be applied in the next startup of the affected programs. To configure the ports that the Denodo embedded Web container listens for incoming connections, click Configure and enter the desired ports in HTTP port number, Shutdown port number and Auxiliary port number. You can also configure these ports using the Virtual DataPort Administration Tool (see section “Configuring the Server” of the Virtual DataPort Administration Guide for information about how to do this). 8.6
UNINSTALLING THE PLATFORM
Click Uninstall to start the uninstall process. You can uninstall all or some of the modules of the Denodo Platform. Before continuing with the uninstallation, stop all Denodo Platform servers and tools (including the Control Center) and close all active instances of Microsoft Internet Explorer.
Figure 21
Denodo Platform Control Center
Uninstalling the Denodo Platform
65
Denodo Platform 6.0
Figure 22
Denodo Platform Control Center
Installation Guide
Processes running warning
66
Denodo Platform 6.0
8.7
Installation Guide
CONFIGURATION OF THE JVM PARAMETERS FROM THE COMMAND LINE
In a host with graphical support, you can use the Denodo Control Center to change the parameters of the Java Virtual Machine (JVM) used to launch each Denodo server (see section 8.5). However, this is not possible in a “headless” environment (i.e. without graphical support). In this section, you will learn to change these parameters in a host without graphical support. To do this, follow these steps: 1. In the first column of the table below, look for the products whose JVM parameters you want to change. 2. Edit their configuration file: look for its value in the “Configuration file” column. 3. For each file, look for the property of the column “Property to Modify” to set the JVM parameters. Note that the characters “:” has to be escaped with the character “\”. E.g. “-XX\:MaxPermSize\=128m” Product
Configuration File
Property to Modify
/conf/vdp/ VDBConfiguration.proper ties
java.env.DENODO_OPTS_START
ITPilot Browser Pool
\conf\iebr owser\IEBrowserConfigur ation.properties
java.env.DENODO_OPTS_START
ITPilot Verification Server
\conf\main tenance\ConfigurationPa rameters.properties
java.env.DENODO_OPTS_START
Scheduler Server
\conf\sche duler\ConfigurationPara meters.properties
java.env.DENODO_OPTS_START
Aracne Server
\conf\arn\ ConfigurationParameters .properties
java.env.DENODO_OPTS_START
Aracne Index
\conf\arnindex\ConfigurationPara meters.properties
java.env.DENODO_OPTS_START
\resources \apachetomcat\conf\tomcat.prop erties
java.env.DENODO_OPTS_START
Virtual DataPort server ITPilot Wrapper server
Aracne Search Engine server Web Container
Table 2
Properties to modify to change the JVM parameters
4. After saving the changes in the modified /bin/regenerateFiles.sh Denodo Platform Control Center
files,
execute
the
script 67
The next time you launch any of the Denodo servers, they will run with the new parameters.
Denodo Platform 6.0
9
Installation Guide
UNATTENDED INSTALLATION OF THE DENODO PLATFORM AND ITS UPDATES
The following sections explain how to perform an unattended installation of the Denodo Platform and its updates. 9.1
MODIFYING THE DENODO PLATFORM INSTALLER TO INCLUDE THE LAST UPDATE
If by the time you are going to install the Denodo Platform, there is an update available for this version (you can check this in the Denodo Support Site), you can easily modify the installer so it also installs the update automatically. To do this, follow these steps: 1. Download a Denodo Platform installer from the Denodo Support Site. See the section 2.5 to know which installer you need. 2. Download the update you want to install. 3. Decompress the installer and the update. 4. Inside the folder of the installer, create a folder named denodo-update. I.e. denodo-install-6.0\denodo-update 5. Rename the jar file inside the zip file of the update to denodo-update.jar. 6. Copy denodo-update.jar to the denodo-update folder. Thus having this path: denodo-install-6.0\denodo-update\denodo-update.jar 7. Compress the directory denodo-install-6.0 again. When using this modified installer, the installer will automatically install the update after the installation process finishes. 9.2
UNATTENDED INSTALLATION OF THE DENODO PLATFORM
You can automate the installation of the Denodo Platform by generating a response file instead of using the graphical wizard or the command line one. The benefit of doing this is that you can complete several installations on multiple hosts without user intervention. This process has two main steps:
Generating a response file. The process is similar to installing the Denodo Platform from the command line, but the result is a response file instead of an actual installation.
Using this response file, execute the unattended installation.
To perform an unattended installation follow these steps: Unattended Installation of the Denodo Platform and its Updates
68
Denodo Platform 6.0
Installation Guide
1. Download a Denodo Platform installer from the Support Site. The section 2.5 explains which one you should select. If by the time you are doing this, there is an update available, you may be interested in modifying the installer to automatically install the update as well. The section 9.1 explains how to do this. 2. Decompress the downloaded file. 3. Open a command line and execute the following commands. On Windows, launch the command line with the option “Run as administrator” even if you are logged in as an administrator. a. If on Windows: cd denodo-install-6.0 installer_cli.bat generate response_file_6_0.xml b. If on Linux: cd denodo-install-6.0 chmod +x installer_cli.sh installer_cli.sh generate response_file_6_0.xml After following the steps of the wizard, the file response_file_6_0.xml will contain the necessary information to perform the installation. 4. On each host where you want to install the Denodo Platform, decompress the zip of the installer and execute the following commands. This will start the unattended installation: a. If on Windows: cd denodo-install-6.0 installer_cli.bat install --autoinstaller response_file_6_0.xml b. If on Linux: cd denodo-install-6.0 chmod +x installer_cli.sh installer_cli.sh install --autoinstaller response_file_6_0.xml
You can use a response file generated with the Linux installer to perform installations on Windows and vice versa. You will only have to modify the response file to set a path that is valid on the operating system. To change this path, edit the response file and change the value of the property “INSTALL_PATH”. Note that you still have to perform the post-installation tasks described in the section 5 on each installation. Most of them can be scripted as well to make the process faster.
Unattended Installation of the Denodo Platform and its Updates
69
Denodo Platform 6.0
9.3
Installation Guide
UNATTENDED INSTALLATION OF UPDATES AND HOTFIXES
You can install an update or a hotfix unattended. That is, without user intervention and without displaying a GUI. This is useful if you want to install an update or a hotfix using a script or in a host without graphical support. To do this, follow these steps: 1. Close all the Denodo programs and tools that are running on the host where you are installing the update or hotfix; stop all the Denodo servers. 2. Although not mandatory, on the production servers we recommend copying the folder of the Denodo Platform before installing a new update. This copy will allow you to restore quickly the Platform to its previous state, if necessary. 3. Read the RELEASE NOTES file of each product. Pay special attention to the “Post-installation actions” of each product. 4. Open a command line. On Windows, launch the command line with the option “Run as administrator” even if you are logged in as an administrator. 5. Decompress the zip file of the update or hotfix. 6. Execute the following command: java -jar denodo-v55-update-.jar -c This installs the update in the directory. When this installer detects that any Denodo server or tool is running, it will ask for configuration to continue. The installer does this to make sure that the update or the hotfix can be installed correctly. Take this into account if you are using a script to install the same update or hotfix on several hosts.
Unattended Installation of the Denodo Platform and its Updates
70
Denodo Platform 6.0
10
INSTALLING
THE
DENODO
WEB
Installation Guide
APPLICATIONS
AS
WINDOWS
SERVICES To install a Denodo web application as a Windows service, do the following: 1. Start a command prompt as an administrator. 2. cd \bin 3. The following table lists the command you have to execute to install a Denodo web application as a Windows service: Denodo Web Application
Command to Install It as a Windows Service
Aracne administration tool
arn_webadmintool_service.bat install
ITPilot administration tool
itpilot_webadmintool_service.bat install
Scheduler administration tool
scheduler_webadmintool_service.bat install
Diagnostic & Monitoring tool
diagnosticmonitoringtool_service.bat install
Information Self Service tool
selfinformationadmintool_service.bat start
Table 3
Commands to install the Denodo web applications as a Windows service
4. Execute services.msc. This will launch the Windows services wizard. 5. Edit the new services and configure them so they start with the 6. Look for one of the new Denodo services (all of them start with word “Denodo”), right-click it and click Properties. 7. Click the Log on tab. 8. Select This account, enter the user name of the user with which you run the Denodo Platform servers. 9. Repeat steps 6 to 8 to configure the other new Denodo services. To remove the Windows service of one of these applications, execute the same script but with the parameter remove instead of start.
Installing the Denodo Web Applications as Windows Services
71
Denodo Platform 6.0
11
Installation Guide
APPENDIX
11.1 DEFAULT PORTS USED BY THE DENODO PLATFORM MODULES The table below lists the TCP/IP port numbers that each module of the Denodo Platform listens for incoming connections. If these modules are behind a firewall, you must open the appropriate ports in the firewall. If the Denodo Platform runs on Windows, remember to open the ports in Windows Firewall as well. These are the default port numbers. They can be changed during the installation process or later, in the administration tool of each module. Server
Default Port
Virtual DataPort and ITPilot Wrapper Server Server port (Virtual DataPort administration tool and JDBC port)
9999
ODBC port
9996
Auxiliary port
9997
Shutdown port
9998
ITPilot Browser Pool Server port
6001
Shutdown port
6002
Auxiliary port
6003
Initial Microsoft browser port
Internet
Explorer
6100. The Browser Pool listens to this port to communicate with the first opened browser. The consecutive ascending port numbers will be used when additional browsers are requested. For example, if the size of the Browser Pool is set to 30, the Browser Pool will listen in the port range from 6001 to 6030.
ITPilot Verification Server Server port
7001
Shutdown port
7002
Appendix
72
Denodo Platform 6.0
Auxiliary port
Installation Guide
7003
ITPilot PDF Conversion Server Server port
8448
Aracne Server Server port
11000
Auxiliary port
10998
Shutdown port
10999
Aracne Index / Search Engine Server Server port
9000
Auxiliary port
8998
Shutdown port
8999
Scheduler Server Server port
8000
Auxiliary port
7998
Shutdown port
7999
Denodo Platform Web container (Scheduler, ITPilot and Aracne Administration Tools, Virtual DataPort Web services, Information Self Service Tool and Diagnostic & Monitoring Tool) Web container port
9090
Shutdown port
9099
JMX port
9098
Auxiliary JMX port
9097
Table 4
Appendix
Default TCP/IP port numbers opened by the Denodo Platform modules
73
Denodo Platform 6.0
Installation Guide
11.2 LIMITATIONS OF THE DENODO EXPRESS LICENSE The Denodo Express license has some limitation compared to a full license. The main one is that it does not allow using ITPilot nor Aracne. It also imposes other limitations over the Virtual DataPort and Scheduler modules: Limitations of Virtual DataPort with Denodo Express Number of concurrent users
1
Maximum requests
3
number
of
simultaneous
Maximum number of rows returned by a query
10,000
ODBC adapters allowed
Microsoft Access and Excel
Sources not allowed
Multidimensional databases: SAP BI, SAP BW, Mondrian, Microsoft Analytical Services and Oracle Essbase. Denodo Aracne Google Search Custom wrappers The Denodo Browser client cannot be used to retrieve data from CSV, JSON or XML files.
View parameters Importing allowed
extensions
You cannot set view parameters in a derived view. (jars)
Version Control System
is
not
This implies not being able to develop custom connectors to sources, custom policies or custom functions. Disabled. With Denodo Express you cannot store your work in a Version Control System such as Subversion, GIT, etc.
JMS listeners cannot be created Resource Manager
Disabled.
Limitations of Scheduler with Denodo Express Maximum number of jobs Table 5
Appendix
1
Limitations of the Denodo Express license (Virtual DataPort and Scheduler)
74
Denodo Platform 6.0
Installation Guide
11.3 USING KERBEROS AUTHENTICATION IN VIRTUAL DATAPORT WITHOUT JOINING A KERBEROS REALM Virtual DataPort can use the authentication method provided by a Kerberos realm (e.g. a Windows Active Directory domain), even if the server where Virtual DataPort runs does not join this realm. To be able to do this, you have to add some properties to the Denodo configuration scripts. Follow these steps: 1. Open the Denodo Control Center 2. Click Configure 3. Click JVM Options 4. In the Virtual DataPort server / ITPilot wrapper server box, add the following (do not remove the existing content of this field): -Djava.security.krb5.realm= Djava.security.krb5.kdc=[:]+ For example, -Djava.security.krb5.realm=CONTOSO.COM -Djava.security.krb5.kdc=dc01.contoso.com If there is more than one key distribution center (kdc) in your domain, add it to the java.security.krb5.kdc property separated by a colon. For example: -Djava.security.krb5.realm=CONTOSO.COM -Djava.security.krb5.kdc=dc01.contoso.com:dc-02.contoso.com 5. Add the same properties you added in the previous step, to the Virtual DataPort Administration Tool box. Important: perform this last step in all the hosts that run an Administration Tool that need to use Kerberos authentication. Not just in the host where the Virtual DataPort server runs. 6. Restart the Virtual DataPort server and its Administration Tools. If the Virtual DataPort Server is running on a “headless” host (i.e. a host without graphical support), you cannot launch the Control Center. Instead, to set the Kerberos system properties do the following: 1. Edit the /conf/vdp/VDBConfiguration.properties file. 2. Add to the java.env.DENODO_OPTS_START property, the properties java.security.krb5.realm and java.security.krb5.kdc with the values explained above. 3. Execute /bin/regenerateFiles.sh Appendix
75
Denodo Platform 6.0
Installation Guide
4. Restart the Virtual DataPort server. Even if the Server runs in a headless environment, you still have to set these properties in the hosts where the Administration Tools run. Note that you have to run these steps in:
The Virtual DataPort server installations.
The installation of all the Administration Tools that will use Kerberos authentication.
You have to define these system properties in the Java applications that will connect to Virtual DataPort using Kerberos authentication.
After performing these steps, configure the Virtual DataPort server and its clients to use Kerberos authentication. The section “Kerberos Authentication” of the Virtual DataPort Administration Guide explains how to do so. 11.4 USING KERBEROS AUTHENTICATION IN THE INFORMATION SELFSERVICE TOOL WITHOUT JOINING A KERBEROS REALM The Information Self-Service Tool can use the authentication method provided by a Kerberos realm (e.g. a Windows Active Directory domain), even if the server where the Information Self-Service Tool runs does not join this realm. To be able to do this, you have to add some properties to the Denodo configuration scripts. Follow these steps: 1. Open the Denodo Control Center. 2. Click Configure. 3. Click JVM Options. 4. In the Web Container box, add the following (do not remove the existing content of this field): -Djava.security.krb5.realm= Djava.security.krb5.kdc=[:]+ For example, -Djava.security.krb5.realm=CONTOSO.COM -Djava.security.krb5.kdc=dc01.contoso.com If there is more than one key distribution center (kdc) in your domain, add it to the java.security.krb5.kdc property separated by a colon. For example: -Djava.security.krb5.realm=CONTOSO.COM -Djava.security.krb5.kdc=dc01.contoso.com:dc-02.contoso.com
Appendix
76
Denodo Platform 6.0
Installation Guide
5. To apply these changes, stop all the Denodo Platform servers and once they are all stopped, start them again. It is important to stop them all so the Denodo web container is stopped as well. If for example, you leave the Information Self-Service started the web container will not shut down and these changes will not take effect. If the Information Self-Service Tool server is running on a “headless” host (i.e. a host without graphical support), you cannot launch the Control Center. Instead, to set the Kerberos system properties do the following: 1. Edit the file /resources/apache-tomcat/tomcat.properties 2. Add to the java.env.DENODO_OPTS_START property, the properties java.security.krb5.realm and java.security.krb5.kdc with the values explained above. 3. Execute /bin/regenerateFiles.sh 4. To apply these changes, stop all the Denodo Platform servers and once they are all stopped, start them again. It is important to stop them all so the Denodo web container is stopped as well. If for example, you leave the Information Self-Service started the web container will not shut down and these changes will not take effect. As already stated you only have to run these steps in the Information Self-Service Tool server installations. After performing these steps, configure the Information Self-Service Tool to use Kerberos authentication. The section “Kerberos Configuration” of the Information Self-Service Tool Administration Guide explains how to do so.
Appendix
77
Denodo Platform 6.0
Installation Guide
11.5 PROVIDING A KRB5 FILE FOR KERBEROS AUTHENTICATION To use Kerberos authentication, you need a krb5 file (i.e. a Kerberos configuration file) when any of these conditions are met:
The Virtual DataPort server runs on Windows and the host does not belong to a Windows domain.
Or the Information Self-Service Tool server runs on Windows and the host does not belong to a Windows domain.
Or Virtual DataPort runs on Linux.
Or the Information Self-Service Tool server runs on Linux.
Or the Kerberos server does not return “forwardable” tickets by default but it can return them.
If any of these conditions are met, check if there is a krb5 file in the default path of the operating system (see table below). Operating System
Default Path for the krb5 file
Windows
\krb5.ini (the system directory usually is C:\Windows). Note that in Windows, the name of the file is krb5.ini and not krb5.conf.
Linux
/etc/krb5.conf
Solaris
/etc/krb5/krb5.conf
Table 6
Default location of the krb5 file depending on the operating system
If the file exists, make sure it has the property forwardable [libdefaults] section of the file.
=
true in the
If the file does not exist, create it in the default path. The figure below is an example of a krb5 file. [libdefaults] default_realm = CONTOSO.COM forwardable = true [realms] CONTOSO.COM = { kdc = dc-01.contoso.com default_domain = CONTOSO.COM } [domain_realm] .contoso.com = CONTOSO.COM Figure 23 Appendix
Sample krb5 file 78
Denodo Platform 6.0
Installation Guide
With the property forwardable = true, the system will request “forwardable” tickets to the Kerberos server. These tickets can be used by the other applications (in this case, the Virtual DataPort server) to request service tickets on behalf of the user. These service tickets will be used to perform Kerberos requests to other services (e.g. databases) on behalf of the Virtual DataPort client (i.e. the Administration Tool, JDBC clients and ODBC clients). Even if the system requests a forwardable ticket, the Active Directory may be configured to not return forwardable tickets. If this is the case, we are in a scenario called “constrained delegation”. When Virtual DataPort runs on Windows and the host belongs to Windows domain, you do not need to define the krb5 file.
Appendix
79
Denodo Platform 6.0
Installation Guide
11.6 CONFIGURING INTERNET EXPLORER RUNNING UNDER THE LOCAL SYSTEM ACCOUNT We recommend configuring all the Denodo Windows services under a specific user account instead of the local system account. The section 5.4 explains how to do this. However, if you cannot do this and you have to change the default settings of Microsoft Internet Explorer when it runs under the local system account, follow these steps: 1. Download the Microsoft utility suite PsTools (http://technet.microsoft.com/en-us/sysinternals/bb896649) and unzip it. 2. Start the "Interactive Services Detection" Windows service. 3. Use the PsExec utility, included in the PsTools suite, to open a Microsoft Internet Explorer instance on the local system account. To do that, execute the following from a command line (adapting the path of the Microsoft Internet Explorer executable to that of your system): PsExec.exe -s -i 0 "C:\Program Files\Internet Explorer\iexplore.exe" 4. Perform the necessary configuration changes. 5. After closing Microsoft Internet Explorer, the system will be ready to use the ITPilot Browser Pool as a Windows service on the local system account.
Appendix
80
Denodo Platform 6.0
Installation Guide
11.7 LAUNCHING THE DENODO STANDALONE APPLICATIONS IN HIGH DPI DISPLAYS When using a monitor with a high DPI display, Windows will display the Virtual DataPort administration tool and the ITPilot Wrapper Generation Tool with a very tiny font. To avoid this problem you have to:
Modify the Windows Registry to tell Windows to look for an external manifest file.
Create external manifest files.
You only have to do this when running these tools on Windows. To do this, follow these steps: 1. Press Windows Button + R, enter regedit and click OK. 2. Navigate to the following registry key: HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Windows > CurrentVersion > SideBySide 3. After selecting the entry SideBySide on the tree of keys, right-click on the right side of the window and click New > DWORD (32 bit) Value. 4. Type PreferExternalManifest and press Enter. 5. Right-click PreferExternalManifest and click Modify. 6. In the Value Data box, enter 1, select Decimal and click Ok. 7. In the rare case that you are launching the Virtual DataPort administration tool or the ITPilot Wrapper Generation Tool with a 32-bit Java Virtual Machine (JVM) on a 64-bit Windows, continue to the next step. Otherwise, go to step 12. If you are not sure if you are in this scenario, go to step 12 because using a 32-bit JVM on a 64-bit Windows is a rare configuration. 8. Navigate to the following registry key: HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVer sion 9. Right-click CurrentVersion and click New > Key. The name of the new key is SideBySide. 10. After selecting the new entry SideBySide on the tree of keys, on the right side of the window right-click and select New > DWORD (32 bit) Value. 11. In the Value Data box, enter 1, select Decimal and click Ok. 12. Close the Registry Editor. Appendix
81
Denodo Platform 6.0
Installation Guide
13. Download the Windows Sysinternals tool “Sigcheck” (https://technet.microsoft.com/en-us/sysinternals/bb897441.aspx) unzip it in the directory \jre\bin.
and
14. Open a command line and execute the following: cd \jre\bin sigcheck -m java.exe 15. The output of this file looks like: Verified: Signed Signing date: 12:18 PM 4/10/2015 Publisher: Oracle America Company: Oracle Corporation Description: Java(TM) Platform SE binary Product: Java(TM) Platform SE 7 U80 Prod version: 7.0.800.15 File version: 7.0.800.15 MachineType: 64-bit Manifest: ... ... ... Copy the output starting from
