Custom Data &
External Data

A Brief Introduction

Jörg Werner, Xceptance

Types of Data

Data that can be processed by XLT

What you'll get automatically:

  • Transaction data
  • Action data
  • Request data
  • Agent resource usage data
  • Standard event data

What you could get on top:

  • Custom events
  • Custom timers
  • Custom values
  • External data

Custom Events

Your tool to count how often certain things were happening during a load test

  • Used to record noticeable events that aren't really errors
  • Event parameters:
    • Event name
    • Detail message
  • Events are grouped by test scenario
    • total number of occurrences per scenario
    • number of occurrences for each message
EventData data = new EventData("ProductHasNoPrice");	// sets timestamp
data.setTestCaseName("TOrder");
data.setMessage("PID: 12345");
Session.getCurrent().getDataManager().logDataRecord(data);

// even shorter
Session.logEvent("ProductHasNoPrice", "PID: 12345");

timers.csv:

E,ProductHasNoPrice,1456928543182,TOrder,PID: 12345

Custom Timers

  • Used to record measurements of elapsed time
    • runtime [ms]
    • failed flag
  • Shown in the Custom Timers section in the load test report
  • Same level of detail as for other timers
    • Statistics
    • Charts
CustomData data = new CustomData("Foo");	// sets start time

try
{
// do the things you want to measure
}
catch (Throwable t)
{
data.setFailed(true);	// mark the action as failed
throw t;
}
finally
{
data.setRunTime();	// automatically sets elapsed time as runtime
Session.getCurrent().getDataManager().logDataRecord(data);
}

timers.csv:

					C,Foo,1456928543182,1234,true

Custom Values

  • Used to record measurements of arbitrary double values
  • Shown in the Custom Values section in the load test report
  • Almost the same level of detail as for timers
    • Statistics
    • Charts
					
double value = 0.1234;	// determine the value

CustomValue data = new CustomValue("CacheHitRatio");	// also sets timestamp
data.setvalue(value);

Session.getCurrent().getDataManager().logDataRecord(data);

timers.csv:

					V,CacheHitRatio,1456928543182,0.1234

Custom Samplers

  • Mechanism to regularly record measurements during a load test
  • Measured values are typically not related to a test scenario
  • A custom sampler will be run only once in a cluster of agents
  • Makes use of custom values

Custom Samplers

Implement your sampler, ... 

public class MyValueSampler extends AbstractCustomSampler
{
public void initialize()
{
    // initialize
}

public void shutdown()
{
    // clean up
}

public double execute()
{
    // determine the value
    double value = ...;
    
    return value;
}
}
... register it in your test suite settings, ... 
com.xceptance.xlt.customSamplers.1.class = com.acme.MyValueSampler
com.xceptance.xlt.customSamplers.1.name = MyValue
com.xceptance.xlt.customSamplers.1.description = ...
com.xceptance.xlt.customSamplers.1.interval = 1000
com.xceptance.xlt.customSamplers.1.chart.title = My Value
com.xceptance.xlt.customSamplers.1.chart.yAxisTitle = Value
#com.xceptance.xlt.customSamplers.1.property.foo = 123
#com.xceptance.xlt.customSamplers.1.property.bar = abc
...
com.xceptance.xlt.customSamplers.9.class = ...
com.xceptance.xlt.customSamplers.9.name = ...
...

... and XLT will take care to execute it regularly.

External Data

Sometimes client-side measurements are not enough ...

  • Enrich the report with statistics and charts from externally gathered data
  • Do this if you want to
    • have all relevant / critical data in one report
    • help analyzing phenomena
    ... but don't have direct access to the system under test
  • Examples:
    • resource usage data (CPU, mem, disk, net)
    • cache sizes
    • GC overhead
    • ...

External Data

  • Approach
    • During the load test: collect the data using any tool
    • After the load test: let XLT process the data files during report generation time
  • Data file formats
    • Out-of-box: CSV
    • Other formats can be read via custom parsers
  • Entries in the data file must be timestamped
  • Parsing and processing of data files is configured in externaldataconfig.xml

External Data

General structure of the configuration file


<files>
<file source="embedded_00/CustomData/data.csv" encoding="UTF-8" parserClass="com.xceptance.xlt.api.report.external.SimpleCsvParser">

    <headline>Demo CSV report</headline>
    <description>This is a demo report based on data from a CSV file.</description>

    <properties>
        <property key="parser.dateFormat.pattern" value="dd.MM.yyyy HH:mm:ss" />
        <property key="parser.dateFormat.timeZone" value="GMT+0" />
        <property key="parser.csv.separator" value="," />
    </properties>

    <tables>...</tables>

    <charts>...</charts>

</file>
</files>

External Data

How to configure the data tables


<tables>
<table title="CPU Statistics" type="minmaxavg">
    <rows>
        <row valueName="1" title="CPU Temperature" unit="°C" />
        <row valueName="2" title="CPU Usage" unit="%" />
    </rows>
</table>
<table title="Network Statistics" type="minmaxavg">
    <rows>
        <row valueName="3" title="Inbound Network Traffic" unit="KB/s" />
        <row valueName="4" title="Outbound Network Traffic" unit="KB/s" />
    </rows>
</table>
</tables>

External Data

How to configure the charts


<charts>
<chart title="QuadCore CPU" yAxisTitle="CPU Temperature [°C]" yAxisTitle2="CPU Usage [%]">
    <seriesCollection>
        <series valueName="1" title="CPU Temperature" axis="1" color="#00FF00" average="10" averageColor="#008400" />
        <series valueName="2" title="CPU Usage" axis="2" color="#FF0000" average="10" averageColor="#840000" />
    </seriesCollection>
</chart>
<chart title="Network Traffic" yAxisTitle="Throughput [KB/s]">
    <seriesCollection>
        <series valueName="3" title="Inbound Network Traffic" color="#00FF00" average="10" averageColor="#008400" />
        <series valueName="4" title="Outbound Network Traffic" color="#0000FF" average="10" averageColor="#000084" />
    </seriesCollection>
</chart>
</charts>