Раздел: Автоматизация > Практикум > Java Test Runner
JTR - Java Test Runner
There is a link to download the latest version of the JTR - Java Test Runner (Git Sources)
Table of contents:
- Test Automation with JTR
- JTR Test Sample
- Run tests
- Test Results Analysis
- Reporting Server Configuration
Test Automation with JTR
JTR is an extremely flexible tool which can be used by automation test developers. First of all it uses high-capacity script language - Java. The second is that it is very simple in using and configuration. To start working with JTR you need just some experience in Java, knowledge of XML, comfortable IDE, application under test and a big wish for test automation.
From testing side you should have only the following:
- Description of the test case or test scenario from a test case management system
- Id of the test case
- Additional test properties and attributes
From technical side to start writing a new test using JTR you will need the following:
- High level XML description of the test suite and test case
- Attributes and properties described in java properties file
- Java class inherited from AbstractSuite from JTR package com.gramant.jtr
Recommended directory structure:
/test-app /config - directory with environment properties, test suites (*.list) and test cases in xml format /lib - directory with java libraries (jars) /resources - additional resources /results - directory of test run results /src - sources test_intg.bat - exdecutable
JTR Test Sample
Test data
Test case: Verify login to the system.
Test Requirement: All authorized users are able to login to the system
Test data:
| Site Url | Registered Test User/Password | Locked Test User/Password |
|---|---|---|
| http://demo.demosystemundertest.com/ |
ADMIN/ADMIN TESTER/TESTER |
bad / bad |
| ... | ... | ... |
Test Suite XML file
From the description of the test data we will create a suite XML file for the JTR test case:
<?xml version="1.0" encoding="utf-8"?> <suite name="Login" class="TestLoginSample"> <test name="Verify registered user login" group="001" type="smoke"/> <test name="Verify locked user login" group="002" type="functional"/> <test name="Verify empty login" group="003" type="regression"/> </suite>
If you are familiar with XML you will easily understand the structure of the JTR test suite.
Suite Login is implemented in class TestLoginSample. It consists of 3 test cases:
- Verify registered user login
- Verify locked user login
- Verify empty login
Attribute "group" shows a relation between JTR test case and test case in Test Management system.
Attribute "type" shows the type of test case: smoke, functional or regression. (see section Run tests for more details)
Test Properties file
Create a test properties file (e.g. test.properties) in the 'config' directory of your test project and add test data in it.
------------- #test site url site.url= http://demo.demosystemundertest.com/ #registered users registered.user.login1 = ADMIN registered.user.password1 = ADMIN registered.user.login2 = TESTER registered.user.password2 = TESTER #locked user locked.user.login = bad locked.user.password = bad -------------
Test Suite Java Class
Look at the following java implementation of the JTR test suite
public class TestLoginSample extends AbstractSuite {
public TestLoginSample(Object o) {
super(o);
}
@DataGenerator(groups = {"001"})
public static List generateUsers() {
List<User> users = new ArrayList<User>();
users.add(new User(JTR.getProperty("registered.user.login1"), JTR.getProperty("registered.user.password1")));
users.add(new User(JTR.getProperty("registered.user.login2"), JTR.getProperty("registered.user.password2")));
return users;
}
@TestPrecondition(groups = {"001", "002", "003"})
public void openBrowser() {
/*
Code to open a web browser
*/
}
@TestPostcondition(groups = {"001", "002", "003"})
public void closeBrowser() {
/*
Code to close a web browser
*/
}
@Test(groups = {"001"}, firstMethod = true)
public TestResult testRegisteredUserLogin() throws IOException {
User user = (User) testObject;
/*
Code to login by Registered user
*/
if (!login(user)) {
LOG.result("Registered user is not logged in");
return TestResult.FAILED;
}
return TestResult.PASSED;
}
@Test(groups = {"002"}, firstMethod = true)
public void testLockedUserLogin() {
User user = new User(JTR.getProperty("locked.user.login"), JTR.getProperty("locked.user.password"));
/*
Code to login by Locked user.
*/
if (login(user)) {
LOG.assertFail("Locked User is logged in");
}
LOG.result("Locked User is NOT logged in");
}
@Test(groups = {"003"}, firstMethod = true)
public TestResult testEmptyUserLogin() {
User user = new User("", "");
/*
Code to login by Empty user
*/
if (login(user)) {
LOG.errorResult("Empty user is logged in");
return TestResult.FAILED;
}
LOG.result("Empty user is not logged in");
return TestResult.PASSED;
}
private boolean login(User user) {
boolean isOk = true;
LOG.action("Login by " + user.toString());
/*
login with user and check that HomePage is opened
isOk = true; // if user is logged in
isOk = false; // if user is NOT logged in
*/
return isOk;
}
private static class User {
String login;
String password;
private User(String login, String password) {
this.login = login;
this.password = password;
}
public String toString() {
return "User{" +
"login='" + login + '\'' +
", password='" + password + '\'' +
'}';
}
}
}
JTR Method Annotations
Annotation @DataGenerator
The DataGenerator annotation describes the method responsible for generation of the test data. This type of method will be useful if you need to execute the same test but with different test data. In our case with different registered users.
The major requirement is that DataGenerator method should return a List with test data objects (in our case the List of Users).
For calling the generated test data you will need to cast variable testObject to the type of your test data. (in our case to the User): User user = (User) testObject;
Annotation @TestPrecondition
The TestPrecondition annotation shows the method which will be executed in the beginning of the test.
Annotation @TestPostcondition
The TestPrecondition annotation shows the method which will be executed in the end of the test.
Annotation @Test
The TestPrecondition annotation shows the method implemented the test.
The relation between @Test, @TestPrecondition, @TestPostcondition and @DataGenerator is done through the groups array implemented in all these annotations. In our sample you can see that @DataGenerator is implemented only for group "001", so before execution of the "001" test data will be generated. The same is for @TestPrecondition and @TestPostcondition. You see that the same precondition will be executed before tests "001", "002", "003" and the same post-condition after them.
JTR Test Results Handling
These are 2 different ways to handle results of test execution:
- Through the TestResult object
- Through the logger mechanism
As you see from the code example @Test(groups = {"001"}) and @Test(groups = {"003"}) return TestResult object. They implemented the first solution. In this case you will have to control the result of test execution by yourself: return TestResult.PASSED or return TestResult.FAILED (option TestResult.SKIPPED is also possible if some test preconditions are not met)
Test method @Test(groups = {"002"}) implements the second option. It returns always void. It means that if method has worked till the end test result will be PASSED by default. In case if exception is thrown or LOG.assertFailed() is called result will be FAILED.
You can choose and use any of them but we propose to use the second solution because it makes tests more flexible and you will not have to return PASSED of FAILED result all the time.
Suites list file
To run your tests you have to create a simple text file with list of test suites to be run.
If your have only one test suite 'login.xml' and it is saved in "config" directory of you project then your suites.list file will be the following:
suites.list --
config/login.xml
If you have more than one test suite your 'suites.list' should contain a list of paths to test suites xml files (a record for each suite xml)
suites.list --
config/login.xml config/add.xml config/delete.xml config/logout.xml
NOTE 1:Path to the suite xml starts from the working directory of your test project
NOTE 2: To comment a suite xml in suites list file use character: #
Run tests
There are several command line options are used for running the JTR tests:
- props - definition of the test properties file
- suites - definition of the suites list
- output - definition of the results output directory (if output parameter is not defined directory "results" will be created by default)
- testtype - definition of tests to be run by test type (if test type is not defined all tests defined in suites list will be executed)
And now we can run test using the following parameters :
- To run only smoke tests use the following (results will be saved in directory SmokeResults):
com.gramant.jtr.JTRun -props config/test.properties -suites config/suites.list -output SmokeResults -testtype smoke
- To run only regression tests use the following (results will be saved in directory RegressionResults):
com.gramant.jtr.JTRun -props config/test.properties -suites config/suites.list -output RegressionResults -testtype regression
- To run only functional tests use the following (results will be saved in directory FunctionalResults):
com.gramant.jtr.JTRun -props config/test.properties -suites config/suites.list -output FunctionalResults -testtype functional
- To run all tests use the following (results will be saved in the default directory results):
com.gramant.jtr.JTRun -props config/test.properties -suites config/suites.list
NOTE: If you use different from the default results directory you will have to change log4j.xml. Set a new path to the test.log file.
Test Results Analysis
JTR package contains two magic xsl file: run.xsl and suite.xsl. Copy them to the output directory of your test project. To see the test result you will have to open run.xml file in any web browser.
Reporting Server Configuration
In order to use reporting server you have to fill in the following options in the test properties file:
##################### ## Reporting Server Options ##################### # environment.type: TEST, PROD or etc. environment.type = TEST # application environment type: prod and demo // depends on application app.environment.type = demo # office name - usefull in case if you have multiple offices office.name = ProTesting # 1 - SFTP, 2 - FTP, 3 - FILE report.transfer.protocol=3 # result xslt location # empty if your test results are in the same folder with xsl schema's # use local path to xsl if you are going to view reports locally # use http path to xsl if you are going to put results into https server xslt.path= test.run.index.file=run.xml test.run.log.file=test.log report.server.index.file=index.xml report.server=false # reporting server path report.server.path=C:\\reportSrv\\ report.server.todo.path=C:\\reportSrv\\todo\\ # settings for using secure FTP transfer #report.server.path= #report.server.todo.path= #report.server.host= #report.server.port= #report.server.user= #report.server.password=
Good luck in using JTR and have fun with test automation.
