4.6.x

XLT 4.6.6

Script Developer

Newlines in value of type command get lost when (re)loading script (#2705)

Text to be entered into <textarea> elements is usually made up of multiple lines. Although replay and saving of type commands that perform such an action works fine in Script Developer, all white-space characters were normalized upon (re)load of the script, causing all newlines in the type command to be eliminated. This is fixed now.

Loading scripts does not finish sporadically (#2713)

When loading huge test suites containing thousands of scripts, it might happen that this process does not finish at all and Script Developer becomes stuck. This phenomenon could be observed only sporadically and restarting Script Developer did help in some cases. Fixed.

Framework

Empty url() statements break parsing of URL strings from CSS content (#2706)

If XLT is configured to load static content such as images, JavaScript, and CSS files, it parses the received content for url statements used in CSS to specify locations of additional resources to be loaded. Unfortunately, statements that do not specify any URL, but are just empty, e.g. url('') or url(), were not parsed correctly and caused invalid URLs to be loaded. This is fixed now.

Interpreting storeTitle command causes ScriptException (#2714)

The easiest way to execute script test cases outside of Script Developer is to let it generate JUnit wrapper classes and run them inside your favorite IDE (Eclipse, IDEA, etc.). When using this mode, any storeTitle command caused the test case to fail with an exception saying that the name of the variable to which the title should be stored is invalid, regardless of its actual value. This has been fixed and such an exception will now be thrown if and only if the specified variable name is indeed invalid.

Update to latest WebDriver (#2719)

Selenium WebDriver has been updated to v2.53.1. This version fixes a compatibility issue with Firefox 47.

Client performance data stored to directory UnknownUser (#2707)

When running a test case via XltFirefoxDriver either from your IDE or as plain JUnit test, the browser performance data files were written to directory UnknownUser instead of the one named after the test case. Fixed.

Implausible request timing values recorded by Firefox add-on (#2721)

Unfortunately, the request timing values recorded by XLT’s Firefox add-on used to measure client performance were not always correct. For example, the following oddities could be observed:

• connect time was twice as big as the TTFB
• small connect, send, wait (server busy), and receive time, but large TTFB
• incorrect send time (always zero)
• DNS lookup time and blocking time were included in connect time

This is fixed now.

Method sendKeys not working for Chrome in headless mode (#2723)

When using the Chrome/Chromium browser in headless mode (via XltChromeDriver), it will be run with an own Xvfb display. Nevertheless, certain operations (esp. sendKeys()) might trigger errors like the following:

unknown error: an X display is required for keycode conversions, consider using Xvfb

This was because the driver server binary chromedriver needs to be run with a virtual display as well. Fixed now.

XLT 4.6.5

Script Developer

Support Firefox 47 (#2699)

Script Developer has been made compatible with Firefox 47, which was released on June 6, 2016. Script Developer runs on Firefox 38/ESR up to 47 now.

ID field is not prepopulated when editing a script module (#2679)

Since XLT 4.5.0, all scripts can have an ID that is unique across the entire test suite. When editing the details of script module that already has such an ID, it was not shown in the appropriate text field as expected. This has been fixed.

Mouse down does not trigger blur (#2680)

When dispatching a mousedown event (explicitly via mouseDown or implicitly via click) to elements which automatically receive focus, the previously focused element was not blurred. This is fixed now.

Incorrect update of command selection after move (#2682)

Under certain circumstances, moving one or more commands up or down lead to an unexpected command selection afterwards. A similar effect was observed, when copy-pasting or removing commands. This has been fixed.

Copy log messages to clipboard not working on Mac OSX (#2683)

Copying messages from the Log panel to the clipboard by either using the keyboard shortcut Cmd+C or the context-menu item did not work on Mac OSX platform. Fixed.

XLT 4.6.4

Script Developer

Support Firefox 46 (#2662)

Script Developer has been made compatible with Firefox 46 which was released on April 26, 2016. Script Developer runs on Firefox 38/ESR up to 46 now.

Module parameter evaluation dependent on provided expression (#2639)

In case you need some random (unique) data, you can use some of the built-in RANDOM macros, e.g. RANDOM.Number(min, max). With the ${variable} notation, you can use macros in any command, even as value for module parameters. When doing so, it is expected to be evaluated only once so that the parameter has a constant value while the module is being replayed. Unfortunately, this was only the case when all macro expressions were itself parameterized with constant values. In case, they used some test/runtime data, the entire macro statement was evaluated each time the module parameter was used and thus the overall parameter value was not constant any more. This is fixed now.

It is possible to import a test suite more than once (#2674)

Usually, you import an already existing test suite only once. But in case you did so more than once, the test suite was listed in the drop-down multiple times. With the new version, importing an already imported test suite will just select it.

Script explorer view becomes inconsistent when filter is active (#2670)

When development goes on and your test suite has grown to a certain size, it is quite hard to keep track of all your scripts. For this purpose, you can use the text input at the bottom of the explorer view and define a filter. The filter is applied to your scripts and thus refines the view above. According to the selected options, the script’s name, tag, or description is examined for a potential match and the view is updated accordingly.

Unfortunately, the entire view might become inconsistent when such a filter is active and you navigate through the packages, change the sorting direction or reload the entire test suite. This is especially true when using the hierarchical view mode, because a package must not show up in the view when none of its scripts match the given filter AND none of its sub-packages fulfill this condition.

The erroneous behavior is fixed now. Whether you are navigating through the view, re-sort it or reload the entire suite, the explorer view is consistent with respect to the applied filter.

Hierarchical package presentation has issues (#2653, #2664)

In case the hierarchical view mode is active and most of your packages are empty, it could happened that a newly created package showed up twice. Similarly, the default package was not shown at all, or packages that had been expanded before were collapsed afterwards. Please note that none of these issues had an effect on the underlying data structure of your test suite. The hierarchical package view is fixed now.

Information panel is not updated properly when re-selecting a script (#2645)

The Information panel below the editor area shows all relevant information about the currently selected script or command. Switching between explorer and editor view, the update of the information panel was not working properly when selecting an already selected script. This is fixed now.

Improve usability of replay speed slider (#2422)

The two labels Slow and Fast aside the replay speed slider can now be clicked to trigger the appropriate action.

Result Browser

Missing response headers (#2649)

Due to a bug in HtmlUnit, response headers that specify the content length, encoding, or checksum were missing for all requests. Now the Request/Response tab lists all the response headers that were transported.

Result Browser source is shown for requests which associated no response content file (#2663)

All data that is necessary to render the Result Browser is given in JSON format and stored in a single file. Moreover, each response is associated with the appropriate request by specifying a relative path to the file where the response’ content has been stored. In this way it can be displayed in the Result Browser when clicking on the Response Content tab. Due to a bug in the main JavaScript file, the Result Browser’s HTML source was shown as the response content in case such a path was given as the empty string. This is fixed now.

Load Testing

NumberFormatException when creating the report for very short test runs (#2669)

Attempts to create load test reports might fail with an NumberFormatException in case the overall test run was very short. Fixed.

XLT 4.6.3

Script Developer

Support for Firefox 45 (#2616)

Script Developer has been made compatible with Firefox 45 which was released on March 8, 2016. Script Developer runs on Firefox 38/ESR up to 45 now.

Context menu item ‘Delete’ has no effect (#2610)

The context menu of the package explorer contains an item called Delete, which is used to delete the selected packages and scripts. Unfortunately, clicking this item had no effect in XLT Script Developer 4.6.2, while the corresponding keyboard shortcut could still be used to trigger the action. This is fixed now.

Deleting a package in hierarchical mode always deletes all sub-packages (#2611)

When removing one or more script packages, the user will be asked whether or not all sub-packages should be removed as well. However, when using the previously introduced hierarchical view mode, Script Developer did always remove all sub-packages without respecting the user’s decision. Fixed now.

Restrict copy/paste operations to scripts (#2612)

Since copy/paste of entire script packages is not supported yet, these operations have been restricted to work for scripts only. Although you are still able to select one or more script packages along with some scripts, the packages won’t end up in the clipboard upon copy. Moreover, in case no script is selected, but only packages, the Copy context menu item will be disabled and the keyboard shortcut won’t trigger any action.

Now that you have copied one or more scripts to the clipboard, you can paste them to the package of your choice by selecting the package directly or by selecting a single script. In the latter case, the package of the selected script will become the target package of the copied scripts. If you have selected more than one target item, the Paste menu item and its keyboard shortcut will be disabled.

Script packages with sub-packages cannot be renamed in hierarchical view (#2613)

When using the hierarchical view mode, any attempt to rename a script package that has at least one sub-package failed, whereas doing so in flat view mode worked fine. This is fixed now.

Parameter expressions are not resolved when editing the commands of a module (#2615)

Assume you’re editing a script that has at least one module call and you would like to edit some of its commands. Before XLT 4.6.0, any parameter expression in the command’s target or value field was replaced with its value - as known so far - in the Resolved Target or Resolved Value field, respectively. Starting with XLT 4.6.0 this was not the case anymore. Fixed.

Changing module name won’t become effective until script reload (#2618)

In case you edit a module call to change the name of the called module (because it’s name was modified or you’d like to call a different module), the changes did not become entirely effective until you reloaded the script. Fixed.

‘Enabled’ status of some context menu items not updated (#2619)

Some of the items in the script explorer context menu change their ‘enabled’ status depending on whether the corresponding action is allowed/possible or not. For example, when the replay of a test starts, the corresponding script will be locked (tab label turns red) and both context menu items Edit Details and Manage Test Data will be disabled. Once replay has finished, these items should be enabled again, but they weren’t. Fixed.

Typing into file input elements does not fire a change event (#2621)

Uploading one or more files is typically done by submitting a form that has one or more input elements of type file. You ‘choose’ the file to upload by simply typing its absolute file path into the file input. However, although the element’s value changed, no change event was fired to signal the modification. This is fixed now.

Selection not updated on item insertion/removal (#2623)

In script editor tabs, the current item is marked by a selection bar. When inserting new items, they should be selected instead to highlight the change and allow for easy removal, if they were inserted by accident. When deleting an item, the next item should be selected automatically to allow deleting several items in a row. All this worked fine in 4.5.x, but did not in 4.6.x. Fixed.

Framework

Selenium update (#2626)

Selenium was updated to the latest available version 2.53.0.

Allow specifying the test suite home directory (#2627)

Typically, you will use at tool like Eclipse, Ant, Maven, or some other external JUnit test runner to execute your functional XLT tests. To pick up the configuration of your test suite, the XLT framework will search for a config sub-directory in the current working directory. However, if your test suite is not located in the current directory, but outside of it or in a sub-directory, XLT won’t find the configuration and will fall back to default settings which is usually not what you want.

Placing your test suite in a different directory than the current directory of your test runner tool is now supported. You can configure the test suite home directory by either setting the Java system property com.xceptance.xlt.testSuiteHomeDir or the environment variable XLT_TEST_SUITE_HOME_DIR. The system property will take precedence. In case none of the two is defined, the current working directory will be used as before.

Don’t log usages of unsupported CSS rules (#2204)

Modern web application use CSS rules like <code>@keyframes</code> that are currently not supported by HtmlUnit’s CSS parser. This might cause a lot of warnings to be logged when load testing with XLT. In order to avoid polluting your log files, we’ve adjusted the log4j configuration files of all sample test suites shipped with XLT. Now, HtmlUnit’s strict error reporting is able to log fatal errors only, but not warnings. Just copy one of these files into your project’s configuration directory to get clean logs again.

HTML comments are not always respected in light-weight mode (#2240)

Load tests can be performed also in light-weight mode. This means that no DOM tree will be constructed, and all parsing and processing has to be done by applying regular expressions on the plain HTML response. When searching for information on the page, any portion of the HTML source code that has been commented out must of course be ignored. This was not the case when searching for a base tag. Hence, base tags in HTML comments were incorrectly considered for computing the base URL of an HTML page. This is fixed now.

ClassCastException in case of non-HTML responses (#2510)

Sometimes the server might not respond with a valid HTML page, as expected, but with content of other types, for example with a plain-text 404 error page. In such cases, a rather technical ClassCastException was thrown by the various loadPage* methods of class com.xceptance.xlt.api.actions.AbstractHtmlPageAction as the returned page object was not of the expected type com.gargoylesoftware.htmlunit.html.HtmlPage. Now a more descriptive exception is thrown instead.

Think time simulated also after the last action (#2537)

When interacting with a web application, real users typically take their time after executing an action before they are going to perform the next one. In the context of XLT, this waiting time is called “thinking time”. XLT simulates this thinking time between actions, but it also did so after the very last action in a scenario. This caused the scenario to take longer than expected. Now think times will be simulated between consecutive actions only.

Performance fix in HtmlUnit (#2622)

HtmlUnit v2.19 introduced a small performance issue regarding the internal management of DOM attribute/character change listeners. The fix for this issue has been backported, so XLT users can benefit from it as well.

Last action listed twice in the result browser (#2593)

The result browser shows the last n (or even all) actions executed during a test scenario. However, for test cases that use XltDriver, the very last action was listed twice in the result browser. Fixed.

TypeAndWait does not submit the form (#2629)

Replaying the command typeAndWait in XLT should not only type some text into an input element, but should also submit the form to which this input element belongs. Unfortunately, the form was submitted for interpreted XML script test cases only, but not for script test cases exported to Java. Fixed now.

XLT 4.6.2

Script Developer

Selecting script package in hierarchical view does not select scripts of sub-packages (#2595)

Before XLT 4.6.0 was out, all scripts and script packages were shown in a flat view only, and when you select a script package, then only the scripts in the selected package were selected. So far, so good. But when the new hierarchical representation mode comes into play, the selection should also include scripts contained in sub-packages of the selected package. Unfortunately, this was not the case and the selection included only the scripts of the selected package. This is fixed now.

Command ‘typeAndWait’ does not submit form (#2596)

Performing a search is one typical use case for the typeAndWait command: you enter some search phrase into a text field and press Enter. However, replaying such a typeAndWait command in XLT Script Developer 4.6.1 entered the search phrase and waited for a page load. But one essential detail was missing: namely pressing the Enter key that submits the form and triggers the page load. Fixed.

Extracting commands into script module causes UI to break (#2597)

Script modules can be created in various ways. One of them is to select one or more commands and extract them into a separate script module that is to be newly created. In XLT Script Developer 4.6.0/1 this caused the UI to be somewhat broken: the selected commands were still shown in the testcase’s editor and deleting them caused a removal of these commands in the newly created script module. The only way to recover from this state was a reload of the entire test suite. Fixed.

Replay of ‘selectAndWait’ command does not work anymore (#2598)

Replaying the command selectAndWait failed in XLT Script Developer 4.6.0/1 with the error message: “Don’t know how to handle command ‘selectAndWait’.” XLT 4.6.2 fixes this issue.

Script Developer keeps on waiting when page load is missing (#2599)

The command timeout that you can configure in the Settings dialog was not effective when waiting for a page load to happen. As a consequence, Script Developer did not only wait longer than it should, but it waited infinitely in case the page load did not happen at all. This is fixed now.

Script Developer hangs when trying to type into file input (#2600)

The ability to type into input elements of type file is essential for testing file upload functionalities. Unfortunately, this ability got lost by accident in XLT 4.6.0 and each attempt to do so caused Script Developer to hang. But no need to worry as XLT 4.6.2 restores this ability completely.

Legal element locator expression is marked as invalid (#2601)

HTML elements often have more than one CSS class assigned, and in case one of these classes contains single quotes then element locator expressions, although totally legal, might be marked as invalid when using them in the Command Details dialog. This is because Script Developer tries to construct alternative locator expressions for the found element and this locator construction algorithm was not able to handle such “special” CSS classes correctly. This is fixed now.

Frame context is lost when replay is paused (#2603)

In case your application under test makes use of one or more frames, you might have already encountered the issue that proceeding a paused replay used the top-level window instead of the currently selected frame window. This was because the internal frame state - which gets lost when replay is paused - was not restored when proceeding replay. The missing restore has been added and replaying your scripts - whether paused or not - should work as expected now.

Package refactoring does not update includes of affected modules (#2609)

Consider the case that you have a test script that calls a module which itself resides in an arbitrary package. When renaming this package to something different (not just the case of the name’s letters) then the test script contained a broken include of the module although the package renaming was successful. The same effect was visible when moving the module into another package: the module landed in the correct package, but the include was broken in the test script. This is fixed now and all refactoring operations don’t lead to broken module includes anymore.

Usability Improvements

New Keyboard Shortcuts for MacOSX Users

When using Script Developer on MacOSX, some of the existing keyboard shortcuts cannot be used at all or only when using an external keyboard. For this reason, Script Developer provides some new shortcuts that replace the existing ones to improve the entire usability for MacOSX users. Users of other platforms are not affected.

In order to insert a new command at the currently selected line press Cmd+I, and in case you want to add it after this line, just press Shift+Cmd+I. These two new combinations replace the usual Insert and Shift+Insert ones that are still effective on non-MacOSX platforms.

To remove commands, scripts, or entire script packages, select the items in question and press Cmd+Backspace, as an alternative to Delete. This new shortcut is needed since there is no Delete key on Apple notebooks, such as the MacBook Air for example.

More Effective Command Filtering

In the Command Details editor, you may not only select the command from the long drop-down list, but you can also start typing the command name to show only those commands that match your filter text. The filtering logic has been changed to perform a contains check now instead of a starts-with check. This allows to enter shorter, but more specific substrings to filter the list of commands more effectively.

Framework

Replacing test data set programmatically not possible (#2605)

The method AbstractTestCase.setTestDataSet(Map) is called by the XLT framework to inject the current test data set in data-driven tests, i.e. tests that are automatically executed repeatedly with different test data sets read from an external file. If you need to programmatically replace the external test data with your own, you can call the method setTestDataSet(Map) in the constructor of your test case. In this case, the framework won’t inject the test data set from the external file any longer which otherwise would have overwritten your test data again.

HTML pages in the result browser looked broken (#2606)

The result browser renders any visited HTML page as it would have looked in a real browser. However, sometimes the rendered pages looked broken because embedded resources such as images or CSS files were empty. This happened for cacheable resources that needed to be revalidated in the course of the test case. If the server responded with status code 304 (Not Modified) for a resource, the empty response body was nevertheless used to “update” the result browser files. Fixed now.

XLT 4.6.1

Script Developer

Assertions are generated twice (#2583)

Although strongly discouraged, your scripts might contain actions that don’t interact with the current HTML page in any way, but only perform some assertions. Then, exporting these actions to XLT’s action API results in Java classes that contain the generated code for all of the action’s assertions twice: in the instance methods preValidate() and postValidate(). This is fixed now.

Framework

Selenium updated (#2592)

Selenium has been updated to v2.52.0.

Load Testing

Test suite upload fails with a recent Java 8 (#2590)

When using Java 1.8.0_72 or later on the agent machines, uploading a test suite failed with the error Premature EOF:

Uploading test suite…  
	Preparing: 0% … 10% … 20% … 30% … 50% … 60% … 70% … 80% … - OK  
	Uploading: 0% … 20% … 40% … - FAILED!

-> Agent Controller ac001 <https://localhost:8500> - Premature EOF

The reason is that with this Java version the implementation of java.net.URL was changed. This caused problems when serializing/deserializing URL objects which is used when the master controller communicates with the agent controllers. Fixed now.

XLT 4.6.0

Release Overview

XLT 4.6.0 comes with many new and improved functionalities. The first improvement … well, you are looking at it right now! We believe that the new documentation structure and design makes the documentation more accessible and more fun to read. And if you miss something in the docs, don’t forget to visit our new XLT Community Forum to ask questions or discuss things with other users.

But there is much more in the box. Script Developer now provides a long-long-awaited feature, undo and redo. From now on, editing scripts will be much more fun. Another just-as-long-awaited feature is to execute parts of a script under certain circumstances only. This is possible now as Script Developer supports conditional module calls. Just define a condition and your module will be executed only if the condition is met.

It goes without saying that the XLT framework supports conditional module calls as well when replaying script test cases. Same as for Firefox, the framework now provides a specialized Web driver for Chrome that records request and event timings and lets Chrome run in headless mode. When running script test cases with native browsers, the framework supports some interesting new execution options, such as limiting page loading times and automatic retry of commands in case of stale elements. If you prefer coding test cases in Java and you don’t want an action to last until the next action begins, you can now close the action programmatically at an earlier point. Last but not least, all core libraries have been updated, especially HtmlUnit and WebDriver/Selenium. This ensures a much better browser emulation and brings support for the latest native browsers.

The load test report has been improved as well. Most notably, the report supports filtering of runtime tables and chart lists by simple, yet powerful text filters. Charts can now be capped at a fixed value and different types of charts can have different capping settings. Furthermore, a configurable set of custom percentiles in the load test report may help you to better assess the performance of your system under test.

Users of Amazon’s EC2 cloud will surely find it useful that the ec2_admin tool can now assign a key pair and pass user data when starting new instances. Moreover, instances can now be terminated across regions.

Sounds interesting? Keep on reading and get all the details.

Script Developer

Conditional Module Calls

Ever had the case that you needed to duplicate a whole test case because of that tiny bit of functionality deep inside the script that was different, but you would rather reuse the test case? Didn’t you sometimes wish you could simply skip the execution of a part of the script under certain circumstances? Well, now you can! No need to copy scripts around and have to maintain different test cases any longer.

The heart of the feature is that modules can be called conditionally now. For a simple IF, you would add a module call and specify a condition when to call this module. By adding one or more alternative module calls and specifying proper conditions, you can extend the concept and implement IF/ELSE or even SWITCH constructs.

In any case, the first step is to move the functionality in question to an own module. Next you would add a call to this module to your script. In the module call’s editor window, you can now specify the condition that needs to be met for this module to be executed. The condition is formulated as an arbitrary JavaScript expression. The module will be executed if and only if the given expression evaluates to true. The expression may also reference test data items or runtime variables via the ${…} or @{…} syntax. Even though the expression can be arbitrarily complex, a typical expression will involve string comparisons only as test data items/variables are simple strings.

Conditional Module Calls

Usability Improvements

Undo/Redo

It’s here. At last! Now any script tab maintains an own change history for editing operations. To navigate the history, use the keyboard shortcuts Ctrl+Z for undo and Shift+Ctrl+Z for redo. Happy editing!

Alternative Representation of the Package Hierarchy

By default, Script Developer represents the package hierarchy as a list of package names. Depending on the structure of your hierarchy (broad vs. deep), a package tree would sometimes be more appropriate. That’s why Script Developer now offers a tree as an alternative way to represent the hierarchy. Right-click the package explorer, and from the pop-up menu choose the package presentation mode that suits you best.

Unlinking Script Editor Tabs and the Package Explorer

By default, the current script editor tab and the package explorer view are linked with each other. When you select a script editor tab, the corresponding script will be selected in the package view as well. This way you always have an indication in which package the current script is located. However, if you find the constantly shifting package view annoying, you can now switch off the coupling of the views. To this end, uncheck the Link with Editor menu item in the package view’s context menu.

Ignore/Remove all Breakpoints

During a longer debug session you might have set a lot of breakpoints in your scripts. These can now be removed all at once via the Clear All Breakpoints context menu item, or - much quicker - via keyboard shortcut Shift+B. Additionally, when running a batch test, you can check an option to ignore any set breakpoints during the batch test.

Test Suite Name and Description

Script test suites can now have a custom name (other than the default name, which is the directory name) and also a description. This data is stored to the new project file for script test suites, <testsuite>/suite.xml. When generating the HTML documentation for your script test suite, name and description will be part of it as well.

Multi-Process Firefox

In the near future, Mozilla will change the process model of Firefox from single-process to multi-process (project e10s, or Electrolysis). This means that future versions of Firefox will use separate processes for the browser UI and the web content. At the moment, this feature is highly experimental and is available in Firefox Developer Edition only. Nevertheless, Script Developer has been adapted to support both single-process and multi-process Firefox browsers.

Supported Firefox Versions

Script Developer has been made compatible with the latest available Firefox version (as of January 26, 2016), while outdated versions are not supported any longer. Thus, Script Developer runs on Firefox 38/ESR up to 44 now.

Framework

Measurements with Real Browsers

In a previous release, we introduced an enhanced FirefoxDriver which enables you to record request and browser event timings as seen by the underlying Firefox browser and can optionally run the browser in headless mode. The same functionality is now available for ChromeDriver and Chrome.

In order to use such an enhanced driver with XML script test cases, specify either firefox_clientperformance or chrome_clientperformance as the driver to use in the test suite configuration, and your test case super class will pick it up:

xlt.webDriver = chrome_clientperformance

For plain WebDriver-based test cases you may want to create/set up the driver yourself. You will find the respective driver classes in the XLT API:

• com.xceptance.xlt.api.webdriver.XltChromeDriver
• com.xceptance.xlt.api.webdriver.XltFirefoxDriver

There are also convenient builder classes that simplify parameterizing the driver in case there is no adequate constructor:

FirefoxProfile customProfile = new FirefoxProfile();  
customProfile.setPreference(“…”, “…”);  
XltFirefoxDriver driver =
XltFirefoxDriver.builder().setProfile(customProfile).setHeadless(true).build();

Scripting

The most important feature is, no doubt, that the framework understands and supports conditional module calls when replaying script test cases. But there are more improvements that might come in handy for script test cases that use a native browser.

Page Load/Javascript Timeout

From now on, the general script timeout is also used to limit the page loading time and the execution time of Javascript code in the browser in case the driver supports this feature. This ensures that the driver doesn’t wait any longer than the given script timeout for the browser to respond and to become ready again to accept new commands such as taking a screenshot for error analysis.

Retrying Commands in Case of Stale Elements

In XLT 4.5.6 we introduced a mechanism to retry a script command in case the target element has become “stale” while the command was being executed. Unfortunately, this mechanism was available for XML script test cases only. But from now on you can use it in your exported test cases as well.

Result Browser

Request Filters

The list of requests for a certain action can get rather long. In case you are interested in special requests only, you can now filter the requests by content type, request method, or protocol.

Request Filters in Action

Easier Copying of Response Content

Sometimes you may need to copy the complete text content of a response. This turns out to be more difficult than expected because the usual shortcut Ctrl-A does not only select the response content, but the whole result browser page. The new Select All button on the top comes to the rescue.

Exact Action Timings

In XLT, Actions encapsulate all the steps needed to carry out a certain interaction with the server. So actions are the basic buildings blocks of your test cases. The XLT framework allows different approaches to structure your test cases into actions. One of them is the Action concept which requires your test steps to be modeled as separate action classes (which have to inherit from AbstractAction).

A much simpler approach is to split your test flow into actions by stating where a new action should begin. An action lasts until the next action begins. This approach is used by script test cases, and if you prefer coding Web-driver-based test cases directly in Java you will typically use it as well.

However, with the latter approach the simplification that an action lasts until the next action starts may lead to imprecise action runtimes. This is because the measured action runtime does not only contain the execution time of the actual interaction, but also:

• the time to set up/clean up the action execution,
• the time to run page validations,
• the time to create screenshots, and
• the time to create the result browser.

For precise action runtimes, you can now be more specific where an action ends by calling the new Session.stopAction() method from your Java-based test cases. This method is complemented by the new Session.startAction(String) method which replaces the already existing - and now deprecated - method Session.setWebDriverActionName(String).

See below for the basic sequence of necessary steps in your code:

…  
// set-up steps  
// pre-vaildation steps  
…  
Session.getCurrent().startAction(“RefineBySize”);  
…  
// implement the action: execute the necessary steps to refine by size  
…  
Session.getCurrent().stopAction();  
…  
// post-validation steps  
// clean-up steps

However, when taking exception/state handling into account, and you want to do this, the typical structure has to be as follows:

…  
// set-up steps  
// pre-vaildation steps  
…  
try {  
	Session.getCurrent().startAction(“RefineBySize”);  
	…  
	// implement the action: execute the necessary steps to refine by size  
	…  
}  
catch (Throwable t) {  
	Session.getCurrent().setFailed(true); // this will mark the action as
	failed  
	throw t;  
}  
finally {  
	Session.getCurrent().stopAction();  
}  
…  
// post-validation steps  
// clean-up steps

We suggest to create a helper class that automatically manages starting and stopping the action and hides the try/catch/finally construct. With this helper your code will be somewhat simpler and could look like so:

…  
// set-up steps  
// pre-vaildation steps  
…  
new ActionHelper(“RefineBySize”) {  
	public void execute() throws Throwable {  
		…  
		// implement the action: execute the necessary steps to refine by size  
		…  
	}  
}.run();  
…  
// post-validation steps  
// clean-up steps

Other Improvements

EdgeDriver

Selenium now comes with a Web driver for the new Edge browser in Windows 10. Although the EdgeDriver does not support the complete Web driver API yet, it is time to make this driver available as a new preconfigured alternative. To use this driver in your Web-driver-based test cases, change the test suite configuration as follows:

xlt.webDriver = edge
xlt.webDriver.edge.pathToDriverServer = c:/path/to/MicrosoftWebDriver.exe

As already known from ChromeDriver and InternetExplorerDriver, you will need an additional driver server binary to use EdgeDriver. Download the latest version from Microsoft.

Load Testing

Load Test Report Enhancements

Filtering Data Tables and Charts

The request table in the load test report can get rather long, and the list of charts below the table as well. So finding the information you are looking for may involve a lot of scrolling. But don’t despair, you’ve got filters now! Simply type a filter expression and the table will show only those rows whose names match the filter. Same for the charts. This works not only for requests, but also for transactions, actions, custom timers, and agents.

For advanced filter expressions, you can specify more than one filter substring in the filter input field, separated by space. A table row (and its corresponding chart) remains visible only if the name matches all specified substrings (AND). Note that case does matter. When prefixing a substring with the ‘-’ character, the name must not contain the substring to remain visible. The filter syntax allows OR-ed filter expressions as well. Just use the ‘|’ character to start a new alternative.

Text Filters

Extended Chart Capping Options

The already existing dynamic chart capping at the n-fold of the mean is a useful feature, but sometimes it is desirable to have a fix capping at a certain value instead. This way, it becomes easier to compare charts visually as all charts represent the same value range. On the other hand, you may need to adjust the capping value more often between runs if the performance varies significantly. Such a fix capping value can now be configured as an alternative to a capping factor.

By default, the report generator won’t cap a chart if a capping is not necessary at all, for instance if the maximum value in the chart is far below the configured capping value. But sometimes you may want to “cap” a chart deliberately at that higher value, for instance, to make charts comparable. That’s why you can force a capping now by setting the respective capping mode.

Often it is necessary to have different capping settings for the different types of runtime charts. For instance, the capping value for transaction charts will usually be higher than the one for request charts. Therefore, the capping value/factor/mode can not only be specified at a global level, but also specifically per chart type, thereby overriding the global settings.

For example, to always cap request charts at 5000 milliseconds and all other runtime charts only if needed and at 5 times the mean value your configuration would look as follows:

com.xceptance.xlt.reportgenerator.charts.cappingFactor = 5
com.xceptance.xlt.reportgenerator.charts.cappingValue.requests = 5000  
com.xceptance.xlt.reportgenerator.charts.cappingMode.requests = always

See the User Manual for more information.

Configurable Percentiles

Up to now, the runtime data tables showed just one percentile, the median. However, other percentiles may be of interest as well, especially “at the slow end”. That’s why you can now configure what percentiles will be shown in the report. Since the percentile values are a nice counterpart to the runtime segmentation values, they are presented similarly and next to the runtime segmentation values in an own main column. The now obsolete median column has been removed.

Percentiles

To configure the list of percentiles to be shown, adjust the corresponding property in file <xlt>/config/reportgenerator.properties as needed:

com.xceptance.xlt.reportgenerator.runtimePercentiles = 50, 95, 99, 99.9

Precomputed External Data

To have all relevant information in one place, the report generator allows to include data that has been generated externally. Previously, this was possible for sampled data only, but precomputed data is now supported as well. In contrast to sampled data, any further data processing, such as calculating statistics, does not make sense for precomputed data. The data just needs to be displayed as a nice data table in the load test report.

For example, see the following precomputed data that summarizes the request statistics per application tier:

863,4783678,12333,0.258,Web Servers
663,1146273,11780,1.027,App Servers  
273,516711,47,0.009,Database

After configuring the report generator accordingly, the resulting load test report presents this data as shown below:

Precomputed External Data

See the User Manual for more information on this topic.

Other Improvements

The Errors section now lists the name of the current action for a certain error. No need to manually extract this information from the error trace any longer! If the shown action name is empty, the error must have occurred outside of an action.

The list of URLs that appears when hovering with the mouse over a request name and the list of result browser directories are sorted now. This way finding a certain entry is easier.

ec2_admin

In case you work a lot with Amazon’s Elastic Compute Cloud and use our ec2_admin tool to manage your machine instances, you may find the following improvements useful.

Terminating Instances Across AWS Regions

Running machine instances with the same name tag assigned can now be terminated all at once, even across regions. This saves you going through each region separately. Simply specify multiple regions separated by comma when prompted.

Assigning a Key Pair

In order to be able to log on an EC2 machine, a key pair must have been assigned to the machine during startup. ec2_admin has been enhanced to provide this feature when starting machines. The key pair to use for a certain AWS region can be configured in <xlt>/config/ec2_admin.properties, but can also be specified on the command line.

Passing User Data When Starting Instances

In case your AWS EC2 instances need some custom configuration data (user data, in Amazon speak), you can now specify this data during startup. There are three supported alternatives to pass the user data:

1. Store the user data to a file and pass its name as a command line argument (-uf <file>).
2. Pass the data as a command line argument (-u "...").
3. When prompted by ec2_admin, enter the user data.

Choose the approach that suits you best, however, the first approach will be the one to use in most cases.

Summary of Configuration Options

ec2_admin now summarizes the chosen options for you to verify them before actually starting the instances:

Configuration:  
	AMI 			  : ami-93b1afff - Xceptance XLT 4.6.0 - BETA - 2016/01/14, Java 8, Ubuntu 14.04, 64bit, IPv6, CP  
	Region 			  : eu-central-1  
	Availability zone : <unspecified>  
	Type 			  : c3.2xlarge  
	Count 			  : 5  
	Name 			  : LPT  
	Key-pair 		  : xc-eu-central-1  
	User data 		  : <none>

Do you want to run the instance(s) with the above configuration? [y/n] =\>

Data Center in Seoul

Recently, Amazon opened a new data center in Seoul, Republic of Korea (ap-northeast-2 ). This new data center is now fully supported in ec2_admin.

Tools

XLT Jenkins Plug-in

Previously, the load test suite to be run by the plug-in was expected to be located directly in the root of the job’s workspace, but can now live deeper inside the job’s workspace or even outside. Just configure the path to the test suite, either relative to the workspace or absolute.

If you use Amazon EC2 machines as load generators, the EC2 credentials needed to start/stop machines can now also be maintained directly in Jenkins. In Jenkins, choose Manage Jenkins > Manage Credentials > Add Credentials > XLT AWS Credentials and specify a name and a pair of secret / access key. In your build job’s configuration, you can now select the credentials to be used:

How to create and use AWS credentials

The load test reports will now be generated with links to the result browsers of failed transactions. In addition to any violated success criteria, a build’s summary page will now also show information about the test cases that failed as well as the slowest requests. This often helps to evaluate the results at a glance without having to dig into the load test report.

XLT Proxy Recorder

The recorder has been improved as well. For instance, it supports an upstream proxy now. This feature allows for proper proxy chaining and is needed to use the recorder in environments which are behind a proxy themselves. Furthermore, Proxy Recorder now provides an additional export template for the new YAML-based syntax of the NoCoding test suite.

Incompatible Changes

Java API Changes

This release comes with some API changes which potentially might break your code. In order to support precomputed external data (which can now be strings as well), we had to widen the type of values in a ValueSet from Double to Object. Please review your custom parsers for external data. We also had to make sure that important helper methods in some of our test case base classes are not overridden by accident, which would cause unexpected behavior. See below for a complete list of breaking changes.

Old Script Library Structure Not Supported Any Longer

Before we introduced script packages, the script library was structured into two main subdirectories, testcases and modules. Since script packages have entered the stage, these special directories were still supported for backward compatibility. Now years have passed, so we decided to eventually abandon the support for the old structure, both in Script Developer and the XLT framework. Please use regular script packages instead to separate scripts and modules.