Open System Testing Architecture

TOC PREV NEXT INDEX


Creating and Editing Tests

Test Development

After planning your Test, use Commander to coordinate the Test development process.

The contents and structure of your Test will depend on the type of Test you are conducting, the nature of the system you are targeting and the aims of your performance Test. In its simplest form a Test can consist of just one Task Group running a single Collector or a Script. However, to produce a fully automated performance Test that accurately simulates the test scenario you want, as well as producing the results data required, it is usually necessary to develop a more detailed Test structure.

Script and Collector Tasks are contained by Task Groups in a Test. Task Groups enable you to control when Tasks run and how they operate during a Test-run. A Test can include one or more Collector-based Task Groups, one or more Script-based Task Groups or a combination of both, depending on whether you are developing an HTTP/S load Test or a production monitoring Test. Add Scripts to generate the HTTP/S load levels required against target systems during a Test-run. Add Collectors to monitor and record performance data from Hosts.

The Task Groups that comprise a Test can be enabled or disabled, before or during a Test-run. Disabling the Script-based Task Groups means that no load is generated when the Test is run. This gives you the ability to use the same Test within both load Test and production monitoring scenarios and enables you to directly compare the performance of a target system within these two environments. After you have added the Scripts and Collectors you need and applied the Task Group settings required, the Test is ready to run.

Tests can be run using networked computers on remote Hosts to execute the Task Groups that comprise a Test. Distributing Task Groups across a network enables you to utilize the processing resources of multiple networked computers. It is then possible to run HTTP/S load Tests that generate realistic heavy loads simulating the activity of many users. In order to do this, HTTP/S Load must be installed on each Host and the OpenSTA Name Server must be running and configured to specify the Repository Host for the Test. For more information on configuring the OpenSTA Name Server to run a distributed Test, see Distributed Tests.

During a Test-run you can monitor Task Group replay from within the Test Pane. Then display the results collected after a Test-run is complete to assist you in analyzing and improving the performance of target systems.

See also:

Test Creation

Distributed Tests

The Web Relay Daemon

Running Tests

Test Creation

After you have created the Scripts and Collectors you need, you are ready to create a new Test. Use the right-click menu function associated with the  Tests folder in the Repository Window, or select File > New Test > Tests in the Menu Bar. Give your Test a name, then double-click the new Test , in the Repository Window to open it.

In Commander an open Test is represented as a table which is displayed in the Test Pane. This is the workspace where you can develop the contents of a Test by adding the Scripts and Collectors you need from the Repository. Select them individually working from the Repository Window. Drag and drop them into the Test Pane in the required order.

When you add a Script or Collector to a new row in the Test table, a new Task and Task Group are automatically created. Select a Task Group table cell and apply the settings you require using the Properties Window located at the bottom of the Test Pane to control when a Task Group starts, the Host used to run the Task Group and the load generated by Script-based Task Groups when a Test is run. Click , in the toolbar to hide and display the Properties Window.

There is only a single instance of the Scripts and Collectors you create. They are included in Tests by reference which means they can be included in many different Tests or in the same Test as separate Tasks. Deleting a Test has no affect on the Scripts and Collectors it contains, and similarly, removing Tasks from a Test does not delete them from the Repository. Any changes you make to Scripts and Collectors are immediately reflected in all the Tests that use them. The Scripts and Collectors that you incorporate in a Test can be removed by overwriting them with new selections or by deleting them from the Test Pane.

Make use of the Duplicate a Task Group function to duplicate a Task Group then edit the Task Group Definition to speed up the Test development process. You can duplicate or delete a Task Group by right-clicking on a Task Group and selecting the required popup menu option.

Tests are saved as .TST files and are stored in the Repository. A Test name must be defined according to the rules for OpenSTA Datanames, with the exception that the name can be up to 60 characters long.

As part of the Test creation process you can make use of the single stepping functionality provided to check that your HTTP/S load Tests run as required during replay. For more information see Single Stepping HTTP/S Load Tests.

If you are developing a Test which includes Scripts that run in sequence within the same Task Group you need to model the Scripts for the Task Group to replay correctly when the Test is run. If your WAE uses cookies or issues session identities, then each Script you create will contain a unique identity that has no connection to the other Scripts included in the Task Group. You need to establish a connection between the Scripts by modeling them.

See also:

The Test Pane

Tasks and Task Groups

The Test Development Process

Developing a Modular Test Structure

The Test Pane

Use the Test Pane to create and edit a Test, then apply the Task Group settings you require to control how they behave during a Test-run. Run and monitor the Test-run then display your results for analysis.

The Test Pane is displayed in the Main Window when you open a Test by double-clicking a new Test , or an existing Test , in the Repository Window.

The Test Pane comprises three sections represented by the following tabs:

Test Pane Features

The Configuration tab view of the Test Pane is displayed below:

See also:

Tasks and Task Groups

The Test Development Process

Tasks and Task Groups

Work from the Repository Window to create new Tests and to open existing ones. Its default location is on the left-hand side of the Commander Main Window,

The Repository Window displays the contents of the Repository and functions as a picking list from where you can select the Scripts and Collectors you want to include in a Test. Use it in conjunction with the Configuration tab of the Test Pane to develop the contents of a Test. Select a Script or Collector from the Repository Window then drag and drop it on to a Task column in the Test table.

The Scripts and Collectors you add to a Test are known as Tasks which are contained within Task Groups. When you add a Script or Collector to a new row in the Test table a new Task and Task Group are automatically created. Each Task Group occupies a single row within the Test table and can be one of two types, either Script-based or Collector-based. A Script-based Task Group can incorporate one Script or a sequence of Script Tasks. Collector-based Task Groups contain a single Collector Task.

Task Groups enable you to control the behavior of the Scripts and Collectors they contain during a Test-run. Select a Task Group cell and apply the settings you require using the Properties Window of the Configuration tab, located at the bottom of the Test Pane. Task Group settings control when a Task Group starts, the Host used to run the Task Group and the load generated by Script-based Task Groups when a Test is run.

When you add a Script or Collector to a Test, you can apply the Task Group settings you require or you can accept the default settings and return later to edit your settings.

The Task Group cells in the Test table are dynamically linked to the Properties Window below, select them one at a time to display and edit the associated Task Group settings in the Properties Window.

Select the Start or Host cells in a Task Group row to control the Schedule and Host settings. Script-based Task Groups and the Script Tasks they contain have additional settings associated with them. Select the VUs and Task cells to control the load levels generated when a Test is run.

Use the Disable/Enable a Task Group option to control which Task Groups are executed when a Test is run. This is a useful feature if you want to disable Script-based Task Groups to turn off the HTTP/S load element. The Test can then be used to monitor a target system within a production scenario.

Task Group Settings include:
See also:

Task Group Settings

The Test Development Process

Task Group Settings

Schedule Settings: apply to Script-based and Collector-based Task Groups

Schedule settings enable data to be collected, or an HTTP/S load to be applied, over specific periods by controlling when Task Groups start and stop during a Test run. Click on the Start cell in a Task Group and use the Properties Window to specify your Schedule settings. Once a Test is running, Schedule settings cannot be edited, but they can be overridden manually using the Start and Stop buttons that appear in the Status column of a Task Group.

The default setting for a Task Group to start is when the Test is run. The Scheduled option starts a Task Group after the number of days and at the time you set. The Delayed option starts a Task Group after the period of time you set, relative to when the Test was started.

There are three options for stopping Task Groups. Manually, means that the Task Group will run continuously until you intervene to end it using the Stop button in the Status column of the Task Group that becomes active during a Test run. You can also schedule a Task Group to stop after a fixed period of time and for Script-based Task Groups only you can instruct the Task Group to stop after completing a number of iterations.

For more information, see Edit the Task Group Schedule Settings.

Host Settings: apply to Script-based and Collector-based Task Groups

Specify the Host computer you want to use to run a Task Group during a Test-run. Click on the Host cell in a Task Group and use the Properties Window to select a Host.

For more information, see Select the Host Used to Run a Task Group.

Virtual User Settings: apply to Script-based Task Groups only

The load generated against target Web Application Environments (WAEs) during a Test-run is controlled by adjusting the Virtual User settings. Click on the VUs cell in a Script-based Task Group and use the Properties Window to specify your Virtual User settings.

Specify the number of Virtual Users you want to run the Task Group to control the HTTP/S load generated when the Task Group is run. Logging levels can be set here to specify the amount of HTTP/S performance statistics gathered from the Virtual Users running the Task Group. Select the Generate Timers option to record the time taken to load each Web page specified in a Script by each Virtual User running the Script.

Use the Batch Start Option to ramp up the load you generate by controlling when the Virtual Users you have assigned to a Task Group run. This is achieved by starting batches of Virtual Users at intervals over a period of time, with a delay between the start of each batch period.

For more information, see Specify the Virtual Users Settings for a Script-based Task Group.

Task Settings: apply to Script Tasks only

Edit the Task settings to control how many times a Script is run. Click on a Task cell in a Script-based Task Group and use the Properties Window to specify your Task settings.

You can schedule a Script Task to stop after a fixed period of time or after completing a number of iterations. You can also specify a Fixed or Variable delay to be applied between each Script iteration completed by a Virtual User during a load Test.

For more information, see Edit the Number of Script Iterations and the Delay Between Iterations.

See also:

The Test Development Process

Edit the Task Group Schedule Settings

Select the Host Used to Run a Task Group

Specify the Virtual Users Settings for a Script-based Task Group

Edit the Number of Script Iterations and the Delay Between Iterations

Disable/Enable a Task Group

Tasks and Task Groups

The Test Development Process

The HTTP/S load Test development process typically includes the following procedures:

See also:

Running Tests

Single Stepping HTTP/S Load Tests

Create a Test
  1. In Commander select File > New Test > Tests.
    Or: In the Repository Window, right-click Tests, and select New Test > Tests.
    The Test appears in the Repository Window with a small crossed red circle over the Test icon , indicating that the file has no content. As soon as you open the Test and add a Script or a Collector, the icon changes to reflect this and appears .
  2. In the Repository Window give the Test a name, which must be an OpenSTA Dataname, with the exception that the name can be up to 60 characters long.
  3. Press Return.
    Note: The new Test is saved automatically in the Repository when you switch to a different function or exit from Commander.
See also:

Open a Test

The Test Development Process

Open a Test
  1. In the Repository Window double-click Tests to open the folder and display the Tests contained.
  2. Double-click a new Test , or an existing Test , to open it in the Configuration tab of the Test Pane, within the Commander Main Window. The Configuration tab displays the Test table where you can add Test Tasks and apply Task Group settings.
    Note: You do not have to close an open Test or Collector before opening another Test.
    Only a single Test or Collector can be open at one time. When you open a Test, the Test or Collector that was already open is closed and any changes you made are automatically saved.
    When a Test is open in the Test Pane, the Test icon in the Repository Window appears with a small, yellow lock icon overlaid, . An open Test cannot be renamed or deleted.
    The name of the open Test is displayed in the Commander Title bar.
See also:

Add Scripts to a Test

Running Tests

The Test Development Process

Add Scripts to a Test
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Double-click Scripts, in the Repository Window to open the folder.
  3. Click on a Script then drag it across to the Test Pane and drop it in a row under a Task column as shown below:
    The selected Script, appears in the first empty row under the first Task column in a new Task Group. Additional Scripts can be added in sequence within the same row.
  1. If your Task Group incorporates more than one Script, select the next Script from the Repository Window, then drag and drop it into the same Task Group row under the next empty Task column cell. Repeat this process until your Script sequence is complete.
  2. Note: If the Scripts included in a Task Group target a WAE that uses cookies or issues session identities, you need to model them before the Task Group will replay correctly when the Test is run. Add additional Scripts to a Test in a new Task Group by dragging and dropping them into the next empty row.
    Note: Your changes are saved automatically in the Repository when you switch to a different function or exit from Commander.

See also:

Developing a Modular Test Structure

Edit the Task Group Schedule Settings

Select the Host Used to Run a Task Group

Specify the Virtual Users Settings for a Script-based Task Group

Edit the Number of Script Iterations and the Delay Between Iterations

Disable/Enable a Task Group

Add Collectors to a Test

Distributed Tests

Running Tests

The Test Development Process

Add Collectors to a Test

Note: You can add and run new Task Groups during a Test-run.

  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Double-click Collectors, in the Repository Window to open the folder and display the contents.
  3. Click on a Collector then drag it across to the Test Pane and drop it in an empty row under the Task 1 column, as shown below:
    The Collector you add appears in the first empty row under the first Task column, in a separate Task Group. Collector-based Task Groups can only contain a single Task.
See also:

Edit the Task Group Schedule Settings

Select the Host Used to Run a Task Group

Disable/Enable a Task Group

Distributed Tests

Running Tests

The Test Development Process

Edit the Task Group Schedule Settings
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Click on the Start cell in a Task Group.
    The current Schedule settings are displayed in the Properties Window at the bottom of the Configuration tab. The default setting is for an Immediate start when the Test is run.
  3. In the Start Task Group section of the Properties Window, click to the right of the selection box and choose a start option:
  1. In the Stop Task Group section of the Properties Window, click to the right of the selection box and choose a stop option:
See also:

Select the Host Used to Run a Task Group

Running Tests

The Test Development Process

Select the Host Used to Run a Task Group

Note: Collector-based Task Groups include a Collector which defines a set of data to be recorded from one or more target Hosts during a Test-run. The Host you select in the Test table determines which computer or device will run the Task Group during a Test-run, not the Host from which data is collected.

  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Click on the Host cell, , in a Task Group.
    The current Host settings are displayed in the Properties Window at the bottom of the Configuration tab. The default setting is localhost, which refers to the computer you are currently using.
  3. In the Host Name text box of the Properties Window, enter the name of the Host to run the Task Group. Your settings are then displayed in the Test table.
    Note: The Host you select must have the OpenSTA Name Server installed and running with the Repository Host setting pointing to the local Host. For more information on configuring the Hosts used to run a Test, see Distributed Tests.
    Note: Your changes are saved automatically in the Repository when you switch to a different function in or exit from Commander.
See also:

Specify the Virtual Users Settings for a Script-based Task Group

Running Tests

Distributed Tests

The Test Development Process

Specify the Virtual Users Settings for a Script-based Task Group
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Click on the VUs cell of the Task Group whose Virtual User settings you want to edit. The current Virtual User settings are displayed in the Properties Window at the bottom of the Configuration tab. Use it to help control the load generated during a Test-run by specifying the number of Virtual Users and when they start.
  3. In the Properties Window enter a value in the first text box to specify the total number of Virtual Users for the Task Group, or use to set a value.
  4. Select the Logging level required for the Task Group to control the level of performance statistics and Timers gathered from Virtual Users. Click , and select either:
    Low: Information collected from the first 10 Virtual Users in the Task Group.
    High: Information collected from all the Virtual Users in the Task Group.
    None: No performance statistics or Timers are gathered.
  5. Click the Generate Timers For Each Page check box, to record results data for the time taken to load each Web page specified in the Scripts, for every Virtual User running the Scripts. Timer information is recorded for the duration of the complete Script if the box is checked or unchecked.
  6. Click on the Introduce Virtual Users in batches check box if you want to ramp up the load you generate by controlling when the Virtual Users you have assigned run. This is achieved by starting groups of Virtual Users in user defined batches.
  7. Use the Batch Start Options section to control your Virtual user batch settings.
See also:

Select the Host Used to Run a Task Group

Running Tests

The Test Development Process

Edit the Number of Script Iterations and the Delay Between Iterations
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Click on a Script Task , whose Task settings you want to edit, to display the current Task settings in the Properties Window at the bottom of the Configuration tab.
  3. With a Script Task selected, use the Properties Window to specify how long the Task runs. Click on the Task Termination box and select an option, either:
  1. You can specify a Fixed or Variable delay between each iteration of a Script Task.
    In the Properties Window, click on the Delay Between Each Iteration box and select an option, either:
See also:

Add Scripts to a Test

Running Tests

The Test Development Process

Delete a Script or Collector from a Test
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Select an individual Script or Collector Task from within a Task Group.
  3. Press Delete, or select Test > Delete Selection.
    Note: Deleting a Collector Task removes the Task Group from the Test table, since a Collector-based Task Group can only include a single Collector.
    Note: The Script or Collector you remove is not deleted from the Repository since all Scripts and Collectors are included in Tests by reference.
    Tip: To remove a Script-based Task Group, which contains a sequence of Scripts, right-click on the Task Group and select Delete Task Group.
See also:

Running Tests

The Test Development Process

Duplicate a Task Group
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Right-click on the Task Group you want to copy.
  3. Select Duplicate Task Group from the popup menu.
    A copy of the Task Group is created at the bottom of the Task Group table.
    Edit the Tasks included and the settings of the new Task Group as required.
See also:

Running Tests

The Test Development Process

Disable/Enable a Task Group

Note: When you add a Script or Collector to a Test the default setting is Enabled.

  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. In the Task Group column click on the check box in the cell of the Task Group that you want to disable.
    When the check box is unchecked, Disabled appears in the Status column cell of the Task Group, indicating that the Task Group will not execute when the Test is run.
  3. Enable a disabled Task Group by clicking on the unchecked check box.
    When the check box is checked Enabled appears in the Status column cell of the Task Group.
    Note: The Start and Stop buttons that appear in the Status cell of a Task Group during a Test-run are used to control when a Task Group starts, not the Enable/Disable function.
See also:

Running Tests

The Test Development Process

Delete a Task Group
  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Right-click on the Task Group you want to delete, or hold down the Shift key to select more than one Task Group for deletion.
  3. Select Delete Task Group from the popup menu.
    Note: Deleting a Task Group does not remove the Scripts or Collectors it contains from the Repository because they are included in a Task Group by reference.
    Tip: Delete a Script or Collector from a Test to remove the Task Group from the Test table, since a Collector-based Task Group can only include a single Collector.
See also:

Running Tests

The Test Development Process

Replace a Script or Collector in a Test

Note: Only Scripts can overwrite other Scripts and only Collectors of the same type can be overwritten using the method below.

  1. Open a Test with the Configuration tab of the Test Pane displayed.
  2. Select the Script or Collector you want to add from the Repository Window and drag and drop it on to the Task cell in the Test Pane containing the Script or Collector that you want to replace.
  3. Click Yes in the confirmation dialog box to overwrite the Task.
    Note: The Script or Collector you overwrite is not deleted from the Repository since all Tasks are included in Tests by reference.
    Note: Existing Task Group settings are unaffected when you replace a Task.
    Note: Your changes are saved automatically in the Repository when you switch to a different function in or exit from Commander.
See also:

Running Tests

The Test Development Process

Compile a Test

Before you run a Test you need to compile to check its validity and ensure it will run.
If you have not compiled a Test before you run it, clicking the Test-run button , automatically compiles the Test.

  1. Open a Test.
  2. Click in the toolbar to compile or select Test > Compile Test.
  3. When your Test compiles successfully you are notified with an on-screen message. Click OK in the dialog.
    A Compile Errors dialog appears if compilation is unsuccessful, detailing the nature of the problem.
See also:

Running Tests

The Test Development Process

Save and Close a Test
See also:

Running Tests

The Test Development Process

Rename a Test

Note: An open Test cannot be renamed.

  1. In the Repository Window double-click Tests, to open the folder and display the Tests contained.
  2. Right-click on a Test and select Rename from the menu.
  3. Enter a new name and press Return.
See also:

Running Tests

The Test Development Process

Delete a Test

Note: An open Test cannot be deleted.

  1. In the Repository Window double-click Tests, to open the folder and display the Tests contained.
  2. Right-click on a Test and select Delete, or select a Test and press Delete.
  3. Click Yes to confirm the deletion of the Test from the Repository.
    Note: Collectors are incorporated into a Test by reference, so deleting a Test does not affect the Tasks they contain.
See also:

The Test Development Process

OpenSTA.org
Mailing Lists
Further enquiries
Documentation feedback
CYRANO.com
TOC PREV NEXT INDEX