Mobile Testing Enterprise Release Notes Release 7.0
Mobile Testing Enterprise 7.0
Release Notes
Mobile Testing Enterprise 7.0 February 2015
Copyright Notice Copyright © 1995-2015 Keynote LLC. All rights reserved. THE INFORMATION CONTAINED IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY EXPRESS REPRESENTATIONS OF WARRANTIES. IN ADDITION, KEYNOTE DISCLAIMS ALL IMPLIED REPRESENTATIONS AND WARRANTIES, INCLUDING ANY WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT OF THIRD PARTY INTELLECTURAL PROPERTY RIGHTS. All text and figures included in this publication are the exclusive property of Keynote and may not be copied, reproduced, or used in any way without the express permission in writing of Keynote. Information in this document is subject to change without notice and does not represent a commitment on the part of Keynote. Keynote may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents except as expressly provided in any written license agreement from Keynote. The trademarks or registered trademarks of Keynote LLC in the United States and other countries include Keynote®, DataPulse®, CustomerScope®, Keynote Customer Experience Rankings®, Perspective®, Keynote Red Alert®, Keynote WebEffective®, The Internet Performance Authority®, MyKeynote®, SIGOS®, SITE®, keynote® The Mobile & Internet Performance Authority™, Keynote FlexUse®, Keynote DeviceAnywhere®, DeviceAnywhere®, Keynote DemoAnywhere®, Keynote MonitorAnywhere®. All related trademarks, trade names, logos, characters, design and trade dress are trademarks or registered trademarks of Keynote LLC in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. Please forward any comments or suggestions regarding this document to Keynote Support. Keynote LLC 777 Mariners Island Blvd. San Mateo, CA 94404
ii
Keynote Confidential
Release Notes
Mobile Testing Enterprise 7.0
Contents 1
Overview................................................................................................................. 4
2
New Features ......................................................................................................... 4 2.1
Native Object-Level Scripting on iOS Devices ........................................................................... 4 2.1.1 Recording Object Commands ........................................................................................ 4 2.1.2 Pick Data Command........................................................................................................ 4 2.1.3 Object-Based Scripting Best Practices ........................................................................... 6 2.1.4 Requirements for iOS Native Object Scripting ............................................................ 6
2.2
Java API Functions for Object-Based Scripting .......................................................................... 7
2.3
Single Sign-On ................................................................................................................................ 7
2.4
Selenium Integration ..................................................................................................................... 7
2.5
Jenkins Integration ......................................................................................................................... 8
3
Updates ................................................................................................................... 9
4
Resolved Issues .................................................................................................... 13
5
Known Issues ....................................................................................................... 13 5.1
End-of-Life Features .................................................................................................................... 15
Keynote Confidential
iii
Mobile Testing Enterprise 7.0
1
Release Notes
Overview
DeviceAnywhere Enterprise is now called Mobile Testing Enterprise from release 7.0 onwards. Release 7.0 of Mobile Testing Enterprise is for standalone deployments as well as the shared system. Object-level scripting on iOS devices is now available in the shared system, with a new data picker command. Integrations with Jenkins and Selenium are now generally available. We also support single sign-on integrating enterprise login systems with Studio and the Enterprise Portal using SAML 2.0. Updates include an upgrade to MPEG-4 format for capturing device video in commands.
2
New Features
2.1 Native Object-Level Scripting on iOS Devices You can now use commands in the Object category to interact directly with objects in native applications on non-jailbroken iOS devices with OS versions 7.1 – 8.1.3. Test applications must be signed to be enabled for object-based scripting (see Requirements below). Object commands can also be used to test web pages in the Safari browser on iOS devices. Object commands are Object Touch, Object Edit, Object Text, and Pick Data; utilities are Launch App and Close App. Figure 2-1 Object Commands
2.1.1 Recording Object Commands Recording of the Object Touch command for native applications is now supported on iOS 7.1+ devices When you opt to record a script, the default recording option is to insert an Object Touch command or use object verification if objects are recognized within the applications. You need to launch the application using Launch App before recording (see Requirements for iOS Native Object Scripting below).
2.1.2 Pick Data Command The new Pick Data command for iOS object scripting enables you to pick data in UIDatePickerView and UIPickerView objects. There is support for multiple fields and different types of strings in pickers, such as dates and other lists. The Pick Data command automatically displays drop-down lists for each field in the picker object such as a date picker (Figure 2-2) and a list picker (Figure 2-3). The command shows a single selected box for the picker object. NOTE The Pick Data command is not supported for Android devices.
4
Keynote Confidential
Release Notes
Mobile Testing Enterprise 7.0
Figure 2-2 Pick Data Command with a Date Picker Multiple fields in date picker for which you can select values
Drop-down list to select value of a field in picker
Date picker on device
Data picker object selected in command
Figure 2-3 Pick Data Command with a List Picker
Selected value for list item
Picker for a list of strings on device
Data picker object selected in command
Keynote Confidential
5
Mobile Testing Enterprise 7.0
Release Notes
2.1.3 Object-Based Scripting Best Practices As a best practice, begin each test case with the Launch App command and keep iOS object scripts short so that each run is less than 30 minutes. If a test case exceeds 30 minutes, launch your application using Launch App within the test case every 30 minutes. This implies that your application is launched several times in a test cycle, depending on the number of test cases it contains and the length of each test case. Depending on your application’s characteristics, relaunching might return it to its home state or require you log in again. These practices prevent memory issues that cause schedules to be interrupted and require the Mac Ensemble Bridge to be restarted.
2.1.4 Requirements for iOS Native Object Scripting Using Object commands on iOS devices is available only for enterprise deployments and requires the following:
iOS 7.1 – 8.1.3 (non-jailbroken)
The DeviceAnywhere Agent version 7.1.3 must be installed on devices.
Keynote’s invoker.ipa application must be installed on devices.
You must sign the test application—shared environment customers can use the application upload utility in Studio with the Sign & Enable Application option selected. Enterprise customers must sign the test application with their own certificate and provision it to the target UDID. Using Studio, you can then upload the application to the Keynote repository and deliver it from there to the device using the Upload As Is option of the application upload utility.
Unlike on Android, you can only work with objects in the signed and enabled applications.
Each iOS test device must be connected by data cable to a Mac mini computer—contact Keynote Support for instructions.
In order to test an application using object-based scripts on iOS devices, you must launch it using the Launch App command. Only applications that are signed and enabled are listed for selection (see image below).
Applications that have been signed and enabled can be launched.
6
Keynote Confidential
Release Notes
Mobile Testing Enterprise 7.0
2.2 Java API Functions for Object-Based Scripting New API functions are available for object-based scripting on iOS and Android devices:
listApplicationsOnDevice() returns lists all the applications installed on the device. It takes no parameters and returns a collection of application information.
uninstallApplication() uninstalls a specific application from a device. You must first run listApplicationsOnDevice() to find the application’s unique identifier, and then use it as an argument to uninstallApplication(). To uninstall an application with the ID com.application.test, you would run the call: uninstallApplication(“com.example.test”) This function returns true if uninstallation succeeds.
2.3 Single Sign-On Enterprise customers (with standalone systems) can use their authentication systems to integrate with our tools; users can log in to Mobile Testing Enterprise Studio and the Enterprise Portal using their existing corporate credentials. Keynote supports service-provider initiated SAML 2.0 requests and IdP-initiated SAML 2.0 requests.
2.4 Selenium Integration Users of Mobile Testing Enterprise Automation in enterprise environments can avail of integration with Selenium (Selendroid JAR version 0.12.0 for mobile browser automation on Android 4.0+ devices using Mobile Testing Enterprise Java API functions. The Mobile Testing Enterprise device.connectAdbTunnel(), device.disconnectAdbTunnel(), and device.forwardAdb() API functions for opening and closing an ADB tunnel to the device and for port forwarding. Requirements/supported integration components (see high-level integration process description below):
Selendroid JAR version 0.12.0 imported in your Studio or Java IDE (e.g., Eclipse) project
Android 4.0+ devices (Multiple Android target API support – 17 to 19)
Writing Selenium scripts in Java
Your test application must be Selendroid supported (see http://selendroid.io/setup.html)
To implement integration with Selendroid: 1
Create a Studio project.
2
Add Selendroid JAR files to the project in Studio: a
Open project properties (right-click project name > Properties).
b
Select the Dependencies tab > Add JAR. The files are installed at the following location: C:\Users\\AppData\Local\DeviceAnywhere\workspaces.\\\lib\user
c
Save project properties.
Keynote Confidential
7
Mobile Testing Enterprise 7.0
Release Notes
3
Call the project into your Java IDE, e.g., Eclipse.
4
Create your Selendroid test case.
5
Use the device.connectAdbTunnel(), device.disconnectAdbTunnel(), and device.forwardAdb() API functions for opening and closing an ADB tunnel to the device and for port forwarding.
6
Before executing the script, acquire the device in Studio or call the device using the Java API, providing the device MCD number in your Java test case. You can also schedule your Selendroid script by calling your Java test case in a test cycle.
You can view script execution on your device in Studio. The code snippet below is a sample script on the Selendroid test application for a device. device.connectAdbTunnel(); device.forwardAdb(); SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.12.0"); WebDriver driver = new SelendroidDriver(capa); WebElement inputField = driver.findElement(By.id("my_text_field")); Assert.assertEquals("true", inputField.getAttribute("enabled")); inputField.sendKeys("Selendroid"); Assert.assertEquals("Selendroid", inputField.getText()); driver.close(); device.disconnectforwardAdb();
2.5 Jenkins Integration The Mobile Testing Enterprise extension for Jenkins is available and supported for Jenkins 1.597 on Windows, Linux, and Mac OS. JDK 1.7 should be installed on Windows Jenkins machines. Figure 2-4 Jenkins Extension Configuration
The extension for Jenkins enables you to:
8
Execute test cases without opening up Studio—the project containing the test case must be published so that Jenkins can kick off execution. You can choose the project and the test case to execute from the Jenkins UI.
Keynote Confidential
Release Notes
Mobile Testing Enterprise 7.0
Upload applications available in the Jenkins workspace to the Keynote repository and devices—users must specify the application name and version, type (iOS or Android), whether the application must be signed (iOS) and enabled for object-based scripting, and which files to upload.
NOTES Multi-device Studio test cases cannot be executed via Jenkins. Integration with a Jenkins slave is not supported.
3
Updates
MPEG-4 Video Format
The Toggle Recording and Capture From Device commands now capture device video in MPEG-4 format. Figure 3-1 Capture Commands
Secure Connections
We now support TLS (Transport Level Security) 1.0 instead of SSL 3.0 for secure connections. Reservations
Account administrators can now make reservations for other users in their accounts. They can also change an existing reservation to change the assignee/user. The new User Name field allows administrators to pick users when creating/editing a reservation. NOTE A device reservation cannot be edited if the reservation is less than 30 minutes away.
Keynote Confidential
9
Mobile Testing Enterprise 7.0
Release Notes
Object Command Updates
The Object Touch command allows you to select an object based on displayed text in the object grid or on object ID.
The Object Touch, Object Edit, and Object Extract Text commands have an object Filter that allows you to filter the objects displayed in the grid by class name.
10
Keynote Confidential
Release Notes
Mobile Testing Enterprise 7.0
In addition to the text label and Keynote-assigned index value of the selected object, the object tree of the page, and all extracted text, script results for object commands now show the coordinates of the screen touched when an object is selected at run time.
Objects displayed in the object layout are often shown with “0” when there is no associated text. For example, see the highlighted object in the image below. Users can now select the Use Description flag to true in Advanced options of Object commands.
This setting displays the object description in the grid instead, making it easy to differentiate and select the desired object. The image below shows the same object before and after the Use Description flag is set to true.
Keynote Confidential
11
Mobile Testing Enterprise 7.0
Release Notes
Updated Device Information
The new Ensemble Bridge field in device information (right-click device > Device Info) helps you identify the Mac computer that an iOS device is attached to. Figure 2 Ensemble Bridge Information in Device Information
12
Keynote Confidential
Release Notes
4
Mobile Testing Enterprise 7.0
Resolved Issues
Open Browsers
The Open Browser command will work even if there is no DeviceAnywhere DOM Server (WebTest) server. Native Objects in Wait Event
Native objects selected in Wait Event were not highlighted when you reopened the command, even though the script containing ran correctly. This issue has now been fixed. IE 10 Support
The Enterprise Portal, Result Portal, and Dashboard are now supported on Internet Explorer 10. Users do not need to change to IE 9 compatibility mode. Object Edit Command Results
Studio results for the Object Edit command now correctly display the extracted text as well as actual and expected results for the text to be entered. Selenium ADB Tunnel
The ADB tunnel disconnected after 20-25 minutes of Selenium script execution. This issue has been fixed, and the previous limitation of running scripts with a maximum length of 15 minutes’ no longer applies. Browser Open on iOS Devices
You can now use the Browser Open command on non-jailbroken iOS devices. NOTE The Clear browser cache option is not supported.
5
Known Issues
Advanced Device Filter
When filtering devices in Studio (using the filter icon above the device list), searching by Operating System in combination with another device feature does not work; the device list is not filtered by both criteria. However, filtering by operating system alone works as expected. Figure 5-1 Advanced Device Filtering by Operating System and Another Criterion
Keynote Confidential
13
Mobile Testing Enterprise 7.0
Release Notes
Data Picker
When you check in and check out scripts with saved Pick Data commands, the individual data fields in the picker are not displayed when you reopen the command. However, the saved values are displayed. To view fields in the command again, navigate to the picker on your mobile device and then refresh the object layout. Selecting by Object ID
When you select an object in the Object Touch command and then switch to Use Object ID to identify the object, the information is not correctly saved when you reopen the command. As a workaround, choose Use Text or Use Object ID first and then select the object you want to identify. Check Marks Not Recognized
Check marks on the device screen in object-based applications are not recognized in Object commands. Object Commands in Safari
Using Object commands to test content in Safari on iOS devices is supported. However, if there is a pop-up in the browser window, the test script freezes. As a workaround, you need to re-launch the Keynote invoker application on the device.
When using Object Edit to enter data into a website field using iOS Safari, do not set Clear Text First to true. The command fails to clear preexisting or placeholder text in a website field.
Timeout for Extracting Objects from a Page
The time allowed for extracting objects from an application page when you first drag a command to the script is hard coded. If your page has thousands of objects, objects will not be displayed if the timeout for extraction is exceeded.
If you run repeatedly into timeout issues when working with an application page with thousands of objects, e.g., by dragging over and timing out many commands, you will see an image of the page instead of the object layout in your command. You must restart the Mac Ensemble Bridge to resolve this.
Limitations of Recording Object Touch on Android 4.3 Samsung
When recording the Object Touch command on Android 4.3 Samsung devices, clicks are recorded incorrectly in the application tray after opening it from the home screen. For instance, you might want to record a click to launch device settings from the application tray after opening the tray from the home screen. The click to launch device settings is incorrectly recorded after navigating to the application tray from the home screen. External Version Control Issue
When using external version control for Studio test assets, you must populate custom folders in your projects with scripts. If user-created folders are left empty, they are erased when you close the project containing them. Waiting for Native Objects in Wait Event
If you have multiple branches based on image-, text-, and object-based reference points in Wait Event, objects can take up to 30 seconds to be matched. As a workaround, use a separate Wait Event command for object-based reference points and other types of reference points.
14
Keynote Confidential
Release Notes
Mobile Testing Enterprise 7.0
Creating a Text-Based State Using a Global Variable
After creating a text-based state that waits to verify a text string contained in a global variable, both the device screen captured and the Text to wait for field are empty if you reopen the implementation without acquiring the device. Advanced Wait Image Settings
In Advanced Wait Image Settings (Show Options Editor in Wait Event and Find and Touch), when you compare a base reference image to another library image, using the zoom buttons causes the pink areas of mismatch to disappear. Java Warning Messages in Console
When building or compiling a Java test case, it is safe to ignore warning messages in the Script Output Console that state “major version 51 is newer than 50, the highest major version supported by this compiler.”
5.1 End-of-Life Features Web category commands are no longer supported for web element-based testing. The commands are targeted to be removed in the next release of Mobile Testing Enterprise.
Keynote Confidential
15
