DoxRunner
​
​
About the documentation
All documentation described in DoxRunner is based on Microsoft Word.
​
A solution (or any group of test cases) is represented by a single document - the solution document (sometimes referred to as the Support document). It contains one or more test cases, plus overall information about the performance test solution that isn't contained in any test case. It is similar to a handover document, but it is more enduring, designed to span multiple projects.
​
Although a test case would normally be updated when it's in the solution document, it can be checked-out into its own separate document and can be checked back in if updated by an external or off-shore party.
​
A test case template can be tailored to suit a Solution and is used as the basis for all new test cases.
​
By documenting a script in the test case rather than in the script itself means you only have to type it once. Then you don't need to worry about it every time you re-record the script, unless there are changes. The Process Raw operation will copy details from the test case document into the script automatically for you, format it consistently, and provide initial parameterization.
​
Solution document
The Solution document is the central repository for one or more test cases, plus solution-wide information such as:
-
The Solution description;
-
Transaction rates;
-
Transaction volumes;
-
Business Process descriptions;
-
Solution history;
-
Contacts;
-
Defect criteria and processes;
-
Test cases.
​
...and more.
Test Case
The DoxRunner test case and script operations are test case-centric. The test case drives the scripting and is the major point of communication between the business, the project, and the test team.
It documents test data, parameters, and business process steps, so that a script can be quickly recorded and generated. Unlike a script, the test case is designed to persist from release to release, so the configuration required to move a script from a raw recording to a finished script remains in place with no or few changes, making a team's response to a changing application much faster.
​
Test case template
A test case template comes as a separate document. Tailor it to suit your solution (and document format standards) because it will be used by the Create test case operation.
​
Reverse engineer a test case from a legacy script
If no legacy script exists, then ignore this paragraph. Many parameters can be extracted from a legacy script and incorporated into a new test case using the Reverse Engineer operation. This also saves time.
About the process
The main DoxRunner process is the Process Raw operation. It uses the test case and a raw script to generate a new script that is either fully parameterised and fully functional, or close to it. It saves the scripter a lot of time in what is normally a time-critical phase of a performance test project.
Record a raw script
Use VUGen to record the script using the test case as a guide. If you need to, update the test case as you go.
As shown by the illustration below, the time taken to record a raw script can be reduced substantially when using the DoxRunner Raw Script Recording process:
-
Only record the step number when a start transaction timer is required;
-
Do not record the step transaction name (it's populated from the test case by the Process Raw operation);
-
Do not record the end transaction timer (it's added automatically by the Process Raw operation);
-
Record all transactions in Action.c. That is, do not assign timer names to vuser_init.c or vuser_end.c (they are automatically updated by the Process Raw operation using the Phase column).
​
Example
Assume the documented Timer ID is 06 and the Timer Name is "Press the Apply Button" and the Test Case ID is TES105.
Also assume all requests and responses for Timer ID 06 occur prior to the beginning of the iteration point (that is, it belongs in vuser_init.c)).
​
If the script is recorded in accordance with the above instructions, only the lr_start_transaction will appear in the script, it will appear in Action.c, and it will appear simply as: lr_start_transaction("6");
​
The Process Raw operation will change it to: lr_start_transaction("TES105 06 Press the Apply Button");
​
...and insert an lr_end_transaction immediately prior to the next lr_start_transaction...
...and move it into vuser_init.c.
​
Pre-pending the Test Case ID improves the format of the Analysis report when the test is complete.
​
Point and click parameterization
When initiated, the Process Raw operation will prepare the green screen shown below.
A list of name-value pairs identified from the raw script is displayed on the left.
Seven parameter rule categories from the test case are displayed on the right.
Associate as many name-value pairs to as many parameter rules as you can.
Add more parameter rules if required - the test case will be updated accordingly.
Delete any parameter rules that are no longer required - the test case will be updated accordingly.
Process raw operation
The Process Raw operation is the key script efficiency enabler.
It combines the documentation and a raw script to produce an updated script that is parameterised, formatted, and documented in accordance with the test case documentation.
​
About the result
Applying the Process Raw operation to a newly recorded raw script will result in the following actions:
-
Safety
-
A copy of the raw script is placed in a sub-folder of the script called ArchivedByVUGenScriptSupport;
-
When finished, a copy of the updated script is also placed in this folder (in case subsequent manual updates cause serious errors).
-
-
Parameterisation
-
Values across the whole script are automatically parameterised according to the Custom Parameters section(s);
-
Values selected by the scripter are automatically parameterised according to the Text Data File section(s);
-
Values selected by the scripter are automatically parameterised according to the LoadTest Database section(s);
-
Values selected by the scripter are automatically parameterised according to the Date/Time section(s);
- Values selected by the scripter are automatically parameterised according to the Additional Attributes section(s);
-
Values selected by the scripter are automatically parameterised according to the Random Numbers section(s);
-
Values selected by the scripter are automatically parameterised according to the VTS section(s);
-
Rules from the Correlation Rules section(s) are automatically applied and parameterised.
-
-
Runtime Settings
-
Rules from the Additional Attributes section are automatically configured in the RunTime Settings and converted to a parameter in vuser_init;
-
The Think Time range as specified in the Configuration section is automatically applied.
-
-
Data
-
Text data files are configured in the script to match the Text Data File rules;
-
The script is configured for any VTS rules;
-
If relevant, the script is instrumented for the LoadTest Database (by creating a DatabaseOperations.c script file)
-
Skeleton functions for the LoadTest Database are prepared in accordance with the LoadTest Database rules
-
-
-
Timers
-
The start transaction timers are converted from the single integer, used during recording (discussed above), to a full name.
-
The full name is formatted using a combination of the Timer Name documented in the Timers section of the test case and the Timer Template CI as specified in the Configuration section;
-
-
End transaction timers are automatically created and inserted after the last request of the relevant transaction timer;
-
The Timers section has a Phase column.
-
Steps with Phase Log In (LI) or To Iteration(TI) are automatically extracted from Action.c so that they are executed when vuser_init is executed;
-
Steps with Phase Iteration (I) remain in Action.c;
-
Steps with Phase Log Out (LO) are automatically extracted from Action.c so that they are executed when vuser_end is executed.
-
-
If selected, a common transaction timer is wrapped around the normal timers. This allows the Analysis process to group similar timers over multiple scripts together for better analysis;
-
If selected, an end to end transaction timer is wrapped around all steps in Action.c.
-
-
Other
-
A web_reg_find function is inserted prior to the first request of a step in accordance with the value defined in the Validation column of the Timers section;
-
All recorded Think Times are removed and replaced by the value defined in the Timers section;
-
If selected, cookies, extra resources, and referrer attributes are commented out;
-
If there is a login function defined in any script file, Session Management elements are configured as follows:
-
A file called SessionManagement.c is created and incorporated into the script;
-
login and logout functions are placed in the SessionManagement.c file;
-
The logout and login functions are called on iteration failure to make it more likely that subsequent iterations start at a known execution point.
-
-
Files in the Included Files folder are copied into the script:
-
If the file has extension of .c it will be configured as an Action;
-
If a file has an extension of .dat or .txt, it will be copied into the script if, and only if, it is represented by a Text Data File rule;
-
Otherwise it will be configured as an Extra File (for example .dll files).
-
-
-
Documentation
-
Sections on the front page of the test case are automatically copied into vuser_init as script documentation;
-
Timers are clearly delineated by a distinctive comment:
-
The distinctive comment is followed by the contents of the Expected Result column from the Timers section;
-
The Expected Result text is followed by text from the NFR section.
-
-
All parameterised values are followed by a comment giving the original value (appended to the end of the line);
-
All parameter assignments are followed by the text from the Description column of the relevant test case table.
-
-
Formatting
-
Long lines of attribute pairs are split into separate lines - one line per pair - for easy visual analysis;
-
Some functions that VUGen places on separate lines are reformatted onto one line;
-
VUGen places a blank line between some functions with the same name - the blank line is removed.
-
​​
About the author
Richard Volzke
B.Ec.
Ph: +614 23 688 077
Mail: Richard_volzke@hotmail.com
Base:
North Ryde
Sydney NSW
Australia
ABN: 96 472 411 400
Richard is an experienced IT professional, proficiant in a number of disciplines, including:
-
Performance Testing
-
Technical Writing
-
ITIL
-
Service Desk
-
IT Management
-
Pre-sales technical support
Richard has experience performance testing using the following protocols:
-
HTTP(S)/HTML
-
Siebel CRM
-
SAP Web / NWBC
-
SAP GUI
-
JSON
-
Microsoft Dynamics CRM
Contact us
Available Services
Our location
Document standards
​
The aim of ISO/IEC/IEEE 29119-3 is to define templates for test documentation that cover the entire software testing life cycle. Each template can be tailored to suit the unique needs of each organisation implementing the standard, to support the standard's implementation within any software development life cycle model. All templates align with the test process defined in ISO/IEC/IEEE 29119-2 and can be produced by applying the processes that are defined in that standard.
​
The well-known and widely used IEEE 829 Test Documentation standard was used as a basis for this standard, and as such, ISO/IEC/IEEE 29119-3 supersedes IEEE 829. Our thanks to the IEEE and its members for their valuable contributions throughout the development of this standard. For more information on IEEE 829 visit http://ieeexplore.ieee.org/.
​
The documents that are defined in ISO/IEC/IEEE 29119-3 are as follows:
​
Organizational Test Process Documentation:
- Test Policy
- Organizational Test Strategy
Test Management Process Documentation:
- Test Plan (including a Test Strategy)
- Test Status Report
- Test Completion Report
Dynamic Test Process Documentation:
- Test Design Specification
- Test Case Specification
- Test Procedure Specification
- Test Data Requirements
- Test Data Readiness Report
- Test Environment Requirements
- Test Environment Readiness Report
- Actual Results
- Test Result
- Test Execution Log
- Test Incident Report