4.0.x

XLT 4.0.5

This section lists and documents all improvements and important fixes for Xceptance LoadTest 4.0.5. Registered customers can access an overview of changes and the latest roadmap at the XLT Information Center.

Improvements

Script Developer supports Firefox 4 (#1220)

Firefox 4 is finally available, so the XLT Script Developer also supports this version now. Currently, Script Developer runs on Firefox v3.5, v3.6, and v4.0.

Test data variables are resolved recursively now (#1237)

When resolving test data variables, the Script Developer and the XLT framework both work recursively now. Thus, if the value of a test data variable ${foo} contains another variable ${bar}, this variable will also be resolved, and so on. This way, variables can be defined based on other variables.

In-place editing of modules (#1239, #1249)

Modules called from other scripts can be expanded to see what they are doing, but editing steps inside the expanded module was not possible. Instead, the module had to be opened explicitly and the user had to locate the step to be modified once again. This process is much more convenient now as the Script Developer will open the module and the respective step editor dialog for you when double-clicking a step in an expanded module. Likewise, steps in the called module can be enabled/disabled from the calling script.

Testing assertions while editing (#1240)

The command editor dialog already provides a Find button to check whether the entered element locator indeed locates the right element on the current page. For assert/waitFor commands, there is an Evaluate button now, which executes the command’s condition with the current page, so the user can immediately check whether the entered text pattern indeed matches a certain text on the page. While not necessary for simple text patterns, this is a big help to get advanced regexp: or glob: text matching patterns right, without the need to let the test case run over and over again.

Bug Fixes

Clicking a child element of an anchor did not trigger the anchor (#1182)

When a child element of an anchor element was clicked (instead of the anchor itself), the framework did not trigger the anchor, i.e. the URL in the anchor’s href attribute was not loaded.

Command assertPageSize always succeeded (#1214)

Replay of the assertPageSize command via the XLT framework succeeded all the time due to an incorrect update of the appropriate counter.

Missing vertical scroll bar in Module Details dialog (#1221)

When editing modules with many parameters some of them were not displayed since there was no vertical scroll bar.

Sometimes wrong XPath element locators recorded (#1222)

When recording clicks on elements contained in anchor elements, the respective XPath locator was not always determined correctly.

Inserting a module by drag&drop placed it at the wrong position (#1223)

When inserting a module via drag&drop to another script, the module was placed at the wrong position if it was dropped right after an expanded module.

Strange script editor behavior (#1226)

The script editor showed weird behavior when scripts still refer to renamed/no longer existing modules. If only one module in the library has such a dangling reference, it was not possible to drag&drop/add a module to another script any more (as the list of available modules in the module call dialog was empty). Furthermore, if a script with a dangling reference was open in a tab, scripts in other tabs could not be edited properly.

Text assertions behaved differently in Script Developer and framework (#1228)

The assertion commands

• assertText
• assertNotText
• waitForText
• waitForNotText

behaved differently in Script Developer and framework when they were used with invisible elements. While the Script Developer assumed the text of an invisible element to always be the empty string, the framework always threw an exception when checking invisible elements. Now the framework uses the same strategy as the Script Developer.

OK button in Module Details dialog disabled (#1230)

Pressing the OK button in the Module Dialog was not possible in some cases (addition/removal of module parameters) due to an incorrect update of the internal error state of the dialog.

Test data management was broken (#1231, #1232, #1236, #1251)

When opening the Test Data Management dialog multiple times in a row, the test data values changed unexpectedly in some cases. For example, placeholders in a value were magically replaced when opening the dialog the second time.

Assertion failure message needs parameter resolution (#1248)

When an assertion fails, the Script Developer displays an appropriate error message. However, if the assertion command contains test data placeholders, they were shown as is, which is not very helpful to understand why the assertion failed. Now any placeholder in the message text is resolved.

XLT 4.0.4

This section lists and documents all important fixes for Xceptance LoadTest 4.0.4. Registered customers can access an overview of changes and the latest roadmap at the XLT Information Center.

Bug Fixes

Double-clicking a module opened the wrong dialog (#1199)

It happened that double-clicking a module call in a script did not open the module call in the editor dialog, but the first command/action in that module.

Base URL not recorded (#1206)

When recording a new test case, the current web site’s base URL was not set at the test case.

Wrong default base URL displayed (#1207)

The displayed default base URL was not always the one associated with the current test case.

Frames not handled correctly when recording (#1211)

When recording interactions on a page that contains at least two frames, the recorded selectFrame commands were not preceded by proper selectWindow commands, so replaying the test case failed as the player tried to locate the frame inside the other frame instead of locating the frame inside the top-level window.

Copy&paste of a command using test data variables is incomplete (#1213)

When copy&pasting a command with externalized test data (e.g. ${userId}) from one script to another script, the command was inserted, but the placeholder did not have a value. Now the placeholder is resolved before copying the command, i.e. the Script Developer will not automatically create a corresponding test data mapping in the new script. If it makes sense to externalize the value for the new script as well, this has to be done explicitly.

XLT 4.0.3

This section lists and documents all important fixes for Xceptance LoadTest 4.0.3. Registered customers can access an overview of changes and the latest roadmap at the XLT Information Center.

Improvements

Replay base URL could not be changed without saving the test script (#1198)

In order to replay an existing test script against another server, one can use the base URL select box to switch to the new server. Formerly, the selected base URL was saved to the script and replaced the original base URL (as determined during recording). Now that URL is handled as a transient override to the base URL stored in the script file when replaying the script. To edit the original/default base URL (and make the changes permanent), use the Test Script Details dialog.

Bug Fixes

Invalid wrapper class code if no package is defined (#1195)

If the default package name field in the Script Developer Settings dialog was left empty, the generated wrapper classes contained an empty package statement (instead of no statement), which does not compile. This has been fixed.

Copy&paste of a command using test data variables is incomplete (#1196)

When copy&pasting a command with externalized test data (e.g. ${userId}) to the same script, the placeholder was copied to the new command, but was not linked to a value. Fixed now.

Implicit waiting for an element broken in Script Developer (#1197)

When trying to locate an element during test case replay, both the Script Developer and the framework repeat the look-up (for only a few times/a very short period) until the element was found, otherwise an error is generated. This implicit waiting process was broken in the Script Developer and is fixed now.

XLT 4.0.2

This section lists and documents all important fixes for Xceptance LoadTest 4.0.2. Registered customers can access an overview of changes and the latest roadmap at the XLT Information Center.

Improvements

Asynchronous loading of script library (#1186)

When Firefox initialized the script developer extension, the script library was loaded into memory as part of the initialization process. Depending on the size of the library, this may have taking a while. During that time, Firefox could not be used. The library is loaded asynchronously now and will not block Firefox. Additionally the loading process has been tuned.

Skip agent controllers with no URL defined (#1193)

The set of participating agent controllers is defined in mastercontroller.properties. To (temporarily) remove agent controllers from this set, it was required to comment out all configuration values for the respective agent controller including the weight. Now it is sufficient to just comment out the line with the (mandatory) url setting.

Bug Fixes

Different values for randomized module parameters (#1137)

When a random value (e.g. ${RANDOM.string(5)}) was passed as a parameter to a script module and used multiple times, a new random value was for each usage. This is fixed now.

No result browser data with exceptions in before/after methods (#1167)

Depending on the test case implementation, significant parts of a test scenario can be located in before methods, usually tagged with the @Before annotation. If an exception occurred inside that method (or in a similar after method), the respective XLT session was not marked as failed. Additionally no result browser data was written to disk and no directory hint was added to the load test report. This has been fixed.

Framework should ignore invisible elements (#1176, #1177, #1178)

When replaying script test cases, the XLT framework did not always ignore invisible elements when trying to locate an element. However, interactions with an invisible element are impossible. An exception will be thrown now and XLT will report an error.

XLT cached AJAX responses (#1188)

When an AJAX request was made multiple times during an action, it was executed only once. For all subsequent invocations, XLT returned the cached response of the first call. Now XLT correctly executes all calls without caching the result.

Only the first test method in a test class was executed (#1192)

With added support for data-driven tests, the code assumed that there is only one test method in each test class. But having just one method is only a requirement for load test cases. There is no reason to enforce this restriction for functional tests. The behavior has been changed.

XLT 4.0.1

This section lists and documents all important fixes for Xceptance LoadTest 4.0.1. Registered customers can access an overview of changes and the latest roadmap at the XLT Information Center.

Bug Fixes

Missing directory information for failed test runs (#1151)

In the Errors section of the load test report one can find (next to an exception) the path to a directory with more information (result browser data). However, the path was only generated for exceptions that originate from an Action class. If the exception was raised from other places (e.g. the test case class), the directory information was missing.

Sometimes interactions are not recorded (#1159)

When recording a test case, the Script Developer did not always recognize an interaction with the current web page. This happened especially for parts of the page which were added later by an AJAX call.

Sometimes interactions are recorded with a wrong target locator (#1160)

When recording a test case, the Script Developer did not always use the actual target element of an interaction, but a parent element.

ProtocolException: Content-Length header already present (#1161)

XLT threw a ProtocolException when retrying the original request in response to an authorization request from the server.

Script replay may fail in Firefox v3.5.x (#1173)

When replaying test cases in Firefox v3.5.x, certain commands failed even though the page was OK. This did not happen if Firefox v3.6.x was used. This issue was caused by using certain Firefox APIs, which are available with v3.6.x only.

XLT 4.0.0

This section lists and documents all new features, improvements, and important fixes of Xceptance LoadTest 4.0.0. Registered customers can access an overview of changes and the latest roadmap at the XLT Information Center.

Features

XLT Script Developer (#702)

As an alternative to writing test cases in Java, you might also use the XLT Script Developer to create script test cases. Script test cases are based on a simple syntax and a reduced set of operations, which makes them a perfect fit for non-programmers. Except the Script Developer, which is an extension to Firefox, no other tool is necessary to create, edit, and manage basic script test cases.

XLT Script Developer

To create a new script test case, the test designer simply uses the application under test. All interactions with the application are recorded in the background and stored to an XML script file as a sequence of script commands. While recording, assertion commands to validate the web pages may be inserted manually. From the Script Developer, script test cases can be replayed in Firefox at any time to quickly check whether the test case still runs successfully.

Existing script test cases can be modified later on, for example, to add new or delete obsolete commands. Common command sequences, which could be reused in other test cases as well, can be refactored to parameterizable script modules. Finally, any recorded value can be extracted out of the script into a test data file to separate test data from script code.

Script files can also be run outside of the browser, via the XLT framework, which simulates a head-less browser. This mode is suitable for unattended test case execution, during functional or load tests. When saving scripts, the Script Developer also creates JUnit test case classes as “wrappers” around script test cases, which serve as a bridge between the XLT framework and the script world. This way, from the framework’s point of view, script test cases are in no way different from test cases written in Java.

Script test cases are strictly linear and provide a reduced set of commands only. However, an XML script test case can also be exported as plain Java code. This opens the world of a real programming language with all its features to realize advanced test scenarios.

In contrast to the Script Recorder in XLT v3.x, the Script Developer does not emit action classes any longer, but generates one Java class for each test case or module. If you prefer structuring your test cases using action classes, you can install and use the old Script Recorder as it runs side-by-side with the new Script Developer in Firefox.

Access to request and response data (#471)

Sometimes a test case needs to check whether a certain request was made in the context of a page and if it were successful, for example, if correct data was sent to a reporting system. Typically, these types of requests do not cause the page’s DOM tree to change, so the only way to check that functionality is to check the requests/responses on the network layer. To better support such advanced validations, XLT provides an API to access network data. The XLT sample test suite testsuite-pebble contains examples how to use this API. See the action classes Login and Homepage.

Improvements

Additional API to log an event (#91)

Now custom events can be logged with a single line of code:

Session.logEvent(eventName, message);

Updated WebDriver and added examples (#627)

The WebDriver library, which is now part of Selenium, has been updated to version 2.0 Alpha 6. Additional XLT demo scenario examples for IE/Chrome/Firefox have been added.

The amount of test result data to be downloaded can be limited (#678)

When a larger load test fails with too many errors, it can be time and bandwidth consuming to download the test results from the agents because of all the error output. Now the user can specify the amount of data to be downloaded by selecting the respective option in a new console menu:

1. measurements, result browsers, and agent log files (i.e. all available data)
2. measurements and result browsers, but no log files
3. measurements only

Integrated JRuby 1.5.1 (#754)

The JRuby libraries have been updated to v1.5.1.

Common JavaDoc for XLT and HtmlUnit (#780)

When hovering over a class or method with the mouse, Eclipse is able to show the corresponding JavaDoc, if the location of the JavaDoc has been configured properly. Since the XLT JAR file contains both XLT and HtmlUnit code, but the respective JavaDoc locations were different, one could either have the JavaDoc of one or the other. Now XLT ships with a common JavaDoc, so Eclipse can show information for both XLT and HtmlUnit at the same time.

Name of test-specific properties file can be passed on the command line (#782)

To choose a certain test configuration, it was necessary to edit the file <testsuite>/config/project.properties to set the property com.xceptance.xlt.testPropertiesFile to point to the right test-specific configuration file. As an alternative to configuration editing, it is now possible to pass the name of the test-specific configuration file on the mastercontroller’s command line:

mastercontroller.sh -auto -report -testPropertiesFile loadtest_1000users.properties

Master controller properties can be overridden with command line options (#784)

In order to a better support a series of automated load tests, it’s possible to override certain properties (usually contained in <xlt>/config/mastercontroller.properties) on the master controller’s command line. For example, two consecutive load tests with different test suites could be run this way now:

./mastercontroller.sh -auto -Dcom.xceptance.xlt.mastercontroller.testSuitePath=samples/testsuite-pebble  
./mastercontroller.sh -auto -Dcom.xceptance.xlt.mastercontroller.testSuitePath=samples/testsuite-showcases

HtmlUnit updated to v2.8 (#881)

The HtmlUnit version integrated into XLT has been updated to v2.8. Go to HtmlUnit 2.8 Release Notes to find out more.

Selecting/listing EC2 instance by tag (#991)

AWS (Amazon Web Services) added the ability to tag EC2 resources to simplify the administration of your cloud infrastructure. As a form of meta data, tags can be used to create user-friendly names and improve coordination between multiple users.

The XLT EC2 administration tool ec2_admin features an additional menu which lets you select your EC2 resources based on the tag name. For example, when listing running instances, you can use filtering to reduce the set of instances shown:

Filter instances by one or more tags:  
(0) <none>  
(1) Name=CustomerA  
(2) Name=CustomerB  
(3) Type=WebServer  
(4) Type=AppServer  
=> 2 4

AJAX call mode configurable (#1088)

HtmlUnit’s web client can be configured whether AJAX calls are to be executed synchronously or asynchronously. By default, XLT configures the web client to always make AJAX calls synchronous if they were started from the test case thread. This way, test case execution waits until the result of the AJAX call is processed, making the test very deterministic and free of wait-for-something-to-appear constructs.

However, under certain circumstances, turning asynchronous AJAX calls into synchronous ones might break expectations in the respective JavaScript code, leading to wrong results. In these cases, we have to execute AJAX calls exactly as requested by the JavaScript programmer. This can be done with a line of code, but unfortunately not in script-based test cases. That’s why the default AJAX behavior is configurable via the framework property com.xceptance.xlt.js.ajax.executionMode now. The property can have the following values:

• resync - synchronous if the AJAX call originates from the test case thread, asynchronous otherwise (the “classic” way)
• normal - as requested in the AJAX call
• sync - always synchronous
• async - always asynchronous

Bug Fixes

This sections covers all important defects that have been fixed with this release.

Not all images referenced by CSS are downloaded (#661)

XLT provides different modes to scan CSS rules for images to download. When using the smart mode “onDemand”, not all images which should be downloaded were actually downloaded. Fixed now.

Styles for output devices are not handled separately (#676)

Some web pages specify separate styles to be used with different output devices. Reading the media attribute, the browser selects the style that should be applied. In HtmlUnit, always all styles were applied. Now only screen media rules are processed to solve this problem.

Credentials in URLs are ignored (#677)

In case credentials are specified as part of a URL, this information will now be used to automatically answer authorization requests (the same way as real browsers do).

Changed caching behavior (#690)

The caching behavior of XLT has been fixed to ensure that non-cacheable resources on a page are not loaded multiple times per action, but only once.

HtmlUnit trims class attributes (#1048)

It can happen that class attributes carry extra whitespace. When recording tests with the Script Developer, you will get an XPath like that:

xpath=/html/body/h1[@class=“Header ”]

Replaying the test via the XLT framework will not longer fail now. Trimming the attribute values was explicitly patched into HtmlUnit because of #491. However, beginning with v3.5, Firefox does not automatically trim the value of the class attribute any longer. Since the minimum version of Firefox XLT supports is v3.5, the fix for #491 is no longer needed.

Result browser does not work with Chrome (#1131)

When viewing result browser files with more recent versions of Chrome, the list of downloaded pages and responses was empty. Fixed now.

API Changes

API changes in XLT 4.0.0 result from changes made by HtmlUnit 2.8. Please consult the HtmlUnit 2.8 Release Notes for the latest changes. Because XLT skipped version 2.6 and 2.7 of HtmlUnit, changes introduced in these versions apply as well.

JDK Compatibility

Beginning with v4.0.0, XLT requires a Java virtual machine 6 or above to run. Java 5 is not supported any longer. The reason is the end-of-life announcement for JDK 5. If you require JDK 5 compatibility, please let us know.