7.0.x

XLT 7.0.0

Version 7 is a major release of XLT. It comes with many new features, but also incompatible changes. Please see below for an overview of what’s in the box.

Because of significant API changes in XLT 7, you will need to migrate your test suites for use with this new version. Please study the section Test Suite Migration carefully.

Overview

Report Generator

The XLT report generator has been tuned extensively with respect to CPU and memory usage. Care has been taken to not copy strings needlessly, thus avoiding a lot of object creations on the heap and hence garbage collection times. More is done in parallel now, using all available CPU cores more efficiently. As a result, report creation is often twice as fast as before. For this to become possible, the API of XLT’s Data classes had to be reworked significantly, especially RequestData. See the section Test Suite Migration below.

The number of distinct URLs for a request with the same name is now estimated instead of being exact. This saves a lot of heap space during report generation, while the error is acceptable.

Load test results containing huge numbers of different events can make the report generator run out of memory. Therefore, the number of events that appear in the report is now limited by the report generator. There are sensible default limits that can also be tailored as needed.

Also note that XLT 7 does no longer support creating reports from load test results that either have been created with XLT < 4.3.0 or contain custom data records with a type code longer than one character. Please use XLT 6 to recreate reports from these result sets.

Test Reports

Report Base

The technical basis of the HTML reports has been updated. This does not only include an update of the third-party JavaScript libraries, but the reports also make much more use of modern CSS now and less of JavaScript. You will need a current browser to view the reports.

The charts are now stored as lossless WEBP images. Compared with PNG, the image sizes are often only a third as large. This helps to save disk space and improves the download times when viewing the report.

Last but not least, chart images are loaded lazily now, so even huge report pages with thousands of charts load faster and become interactable earlier.

Report Design

The general design of the report pages has been modernized, and many details have been improved. For example, the page header is now sticky and therefore always visible. If you configured a project name in your test suite, that name is shown in the header as well. When dealing with multiple projects at the same time, this always-visible information may help to quickly tell which project a report belongs to.

The report pages are also more responsive to different devices now. On wider screens, the font size gets a bit larger, so reading the data tables is easier on the eye. On smaller screens, especially on mobile devices, the navigation bar turns into a hamburger menu.

Some pages, such as Transactions, Actions, and Requests, feature a section with summary charts. These sections have been moved to the top of their pages. This way, they are immediately visible without having to scroll past the data table, which can get lengthy at times.

Additional Report Content

The load and performance test report has been extended to show additional details.

On the Network page, the new section HTTP Request Methods provides some insight into how often each HTTP request method has been used during the test.

The new section IP Addresses on the same page lists the IP addresses that have been contacted during the test and how often. This data is useful if the host name of the system under test resolves to multiple IP addresses. Use this information to learn if the distribution of traffic to these multiple target addresses during the test was as expected.

The Events page now shows how many events of a certain type have been dropped during the creation of the report.

The Agents page got a new tab section with summary charts right above the agent table. Similar to transactions, actions, and requests, these charts contain the data of all agents. Use them to get an idea of the overall behavior of the agents.

Framework

HtmlUnit, a browser emulation for Java, is the heart of the XLT framework. It has been updated to the new major version 3.x. Since there are also some significant changes in the API of HtmlUnit, consult the section Test Suite Migration for guidance.

Almost all other third-party libraries have been updated as well, including Selenium. Since Selenium 4.x does no longer support the “legacy mode” of FirefoxDriver, we had to remove that mode from XLT as well. Also note that OperaDriver has been removed from Selenium altogether.

XLT provides extensions for Chrome and Firefox to record request and response details when testing with these browsers instead of HtmlUnit’s browser emulation. The extension for Chrome has been updated to be compatible with Chrome Manifest v3, which will be mandatory in the near future.

Load Testing

An XLT load test agent searches for Java classes and JAR files at certain locations within the uploaded test suite. The agent now knows new locations, typical for Kotlin-based test suites that use Gradle as the build tool.

Timestamps recorded by XLT are either based on the local time of an agent machine or on the Master Controller’s time (as a reference time for all agents alike). With XLT 7, the new default is the Master Controller’s time.

XLT comes with an error limiter that can reduce the number of result browsers written to disk for errors that have already occurred during a load test. This error limiter is now active by default.

XLT Distribution Archive

The XLT distribution archive xlt-7.0.0.zip is a bit leaner now, as we don’t ship the XLT JavaDocs any longer. Please visit our Web site for the latest version of the JavaDocs.

XLT Images

Xceptance provides ready-to-use public XLT images for AWS and Docker. The images are now based on Debian 11.x (“bullseye”). They come with the latest XLT and JDK 11 and are tuned for load testing. For real-browser load tests, they also contain the latest Chromium and Firefox (ESR) browsers with matching web driver binaries, chromedriver and geckodriver.

If you are looking for XLT images for other clouds, visit XLT’s sister project, XLT Packer. This project does not only contain scripts to install and set up the contents of an XLT image, but also provides Packer templates to actually create such images for the following clouds:

• AWS
• DigitalOcean
• Google
• Hetzner

Follow the instructions in the README to create an XLT image for one of the supported clouds yourself.

Test Suite Migration

As outlined above, XLT 7.x comes with breaking changes. In order to migrate your test suites, you will need to perform the following steps.

1. Fix Compile Errors

HtmlUnit 3

The name of the HtmlUnit root package has changed. Since the rest of the package structure remains as before, migration is straightforward. Simply replace com.gargoylesoftware.htmlunit with the new name org.htmlunit via a global search/replace in your IDE over all your Java source files and also all configuration files, especially log4j2 configuration files.

There has also been a semantic change in HtmlUnit when dealing with the value of <input> elements. Previously, the API only had the getValueAttribute()/setValueAttribute() methods to get or set the value of an input.

HtmlUnit now clearly separates between the internal value of an input element and the value attribute in the HTML source code. To get/set the internal value, you would now need to call the new getValue()/setValue() methods. To get/set the value attribute in the DOM, call getValueAttribute()/setValueAttribute().

Typically, you will now need to call getValue()/setValue() instead of getValueAttribute()/setValueAttribute() in your code. Only if you really intend to get or change the value attribute in the DOM tree should you call the latter methods. Review your code carefully and use the correct methods.

More migration details can be found here.

XLT Framework

The API of XLT’s Data classes, such as RequestData or ActionData, has been changed. Typically, XLT takes care of creating, populating, and writing objects of these classes to the timers.csv files under the hood. So if you don’t use these classes directly, there is nothing for you to do here.

However, if you do log such data objects, for example, when making a custom request or recording a custom timing or a custom value, you will need to migrate your code. This typically involves these steps:

• Many getter methods now return XltCharBuffer instead of String. If you need the string value, call toString() on XltCharBuffer.
• The timestamp of the data record is not set automatically any longer during object creation. You will need to set it explicitly.
• You will need to measure any runtime yourself and set it in the data object.

See below for a complete example of recording a custom timing using the CustomData class. The migration for other data classes is done accordingly.

// create the data object
CustomData data = new CustomData("CustomOperation");
data.setTime(GlobalClock.millis());

// perform the task to be measured
long start = TimerUtils.get().getStartTime();
...
long runTimeMS = TimerUtils.get().getElapsedTime(start);

// set the runtime and log the data record 
data.setRunTime(runTimeMS);
Session.getCurrent().getDataManager().logDataRecord(data);

If you have created custom data classes, please pay attention to these topics:

• The data record type code is no longer a string but a character only. Change the type code to a character that is not used by XLT’s data classes yet (i.e., T, A, R, P, C, V, E, and J).
• Study the implementation of XLT’s data classes to learn how to rewrite the value parsing logic (see parseRemainingValues());

2. Review the Configuration of Your Test Suite

XLT 7 changes the default values of some test suite properties. See file <your-test-suite>/config/default.properties. For example, the Master Controller’s time is now used by default:

com.xceptance.xlt.useMasterControllerTime = true

Furthermore, the result browser limiter is now on by default, and the respective configuration settings have got new names. See below for the default values:

com.xceptance.xlt.output2disk.onError.limiter.maxDifferentErrors = 500
com.xceptance.xlt.output2disk.onError.limiter.maxDumps = 10
com.xceptance.xlt.output2disk.onError.limiter.resetInterval = 60m

Before applying request processing rules (a.k.a. merge rules), XLT 7 now removes any appended indexes from request names by default:

com.xceptance.xlt.reportgenerator.requests.removeIndexes = true

Please check if the new defaults are suitable for you and adjust the values as needed.

3. Test Your Scenarios

Now it’s time to check if everything still works as before. Start by running the test scenarios from your IDE. They should run and finish with the expected results. After that, perform a small load test. Check the results and verify the output in all log files.

4. Review the Report Generator Configuration

As mentioned above, the XLT report generator has learned some new features. You can configure them in the file <xlt>/config/reportgenerator.properties. See below for the properties (and their default values) that configure the event limiter and the WEBP chart generator:

com.xceptance.xlt.reportgenerator.events.eventLimit = 100
com.xceptance.xlt.reportgenerator.events.messageLimit = 100

com.xceptance.xlt.reportgenerator.charts.compressionFactor = 0.0

Again, please check if the default values are suitable for you and adjust the settings as needed.

AWS AMI Retirement Notice

Xceptance provides machine images for Amazon’s Elastic Compute Cloud with XLT installed and configured for use as load agents. As part of our regular clean-up process, we will retire AWS AMIs older than XLT version 6.2.x on September 1st, 2023. If you need an older version, please build your own AMI (see here for more details) or copy the respective AMI to your own AMI registry in time.

As a note for the future: Xceptance will not guarantee that provided AMIs (and, in the future, images for other cloud providers) are available forever. We strongly encourage you to migrate frequently to the most recent version. If you need a particular setup maintained for a longer period of time, please talk to Xceptance and consider a support contract.