How to Use TestNG Listeners

In this article I will explain and show an example of TestNG Listeners. Actually, TestNG provides many types of listeners but we generally use some of them. Thats why I did not give all type of TetsNG Listeners but you can find all of them in this article.

A TestNG listener always extends org.testng.ITestNGListener and TestNG provides us below listener types:

• IExecutionListener
• IAnnotationTransformer
• ISuiteListener
• ITestListener
• IConfigurationListener
• IMethodInterceptor
• IInvokedMethodListener
• IHookable
• IReporter

In this article I will show the most common Listener type  ITestListener.

In our tests we implement ITestListener and use its methods. It has following methods and events:

• onStart is invoked after the test class is instantiated and before any configuration method is called.
• onTestSuccess is invoked on success of a test.
• onTestFailure is invoked on failure of a test.
• onTestSkipped is invoked whenever a test is skipped.
• onTestFailedButWithinSuccessPercentage is invoked each time a method fails but is within the success percentage requested. Don’t worry I will explain it with example ;)
• onFinish is invoked after all the tests have run and all their Configuration methods have been called.

Now, Lets do an example and examine how our Listener works on specific events.

We have three JAVA classes; BaseTest, Listener and SampleTest.

BaseTest:

In BaseTest, I used @BeforeMethod and @AfterMethod annotations. @BeforeMethod is triggered before the test and @AfterMethod is triggered after the test finishes.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

/**
 * Created by ONUR on 11.12.2016.
 */
public class BaseTest {

    @BeforeMethod
    public void BeforeMethod(){
        System.out.println("I am in Before Method! Test is starting!");
    }

    @AfterMethod
    public void AfterMethod(){
        System.out.println("I am in After Method! Test is ending!");

    }
}

SampleTest:

SampleTest is our test class,

test1 passes without any problem.

test2 always fails because it expects RuntimeExceptions but in the test method there is not any runtime exception.

test3 always skipped because it throws SkipException.

test4 is a little bit different than others.

I set %60 success percentage and 5 invocation count.  Thus, it runs 5 times and if 3 times it passes, test method is considered as passed! In the code, test method fails 2 times and passes 3 times so success ratio is (3/5)*100=60 thus we can consider this test method as passed! As you can see below picture, all the 5 tests of test4() method is shown as passed! Because we are in success ratio %60.

But, when we increase the success ratio to %80, this time one of the tests will be shown as failed! Because, %80 success ratio means “4 times tests pass” but our tests pass 3 times. Thus one of them is shown as failed.

Now, let’s set the success ratio to %100 (it is the default ratio). If all test pass, we can reach %100 success ratio but in our test method, 3 of our test pass and two of them fail. Thus, we can not reach %100 success ratio and we will see two failed tests as a result.

import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.Test;


/**
 * Created by ONUR on 11.12.2016.
 */
public class SampleTest extends BaseTest{


    @Test
    public void test1() {
        System.out.println("I am in test1 test method and it will pass.");
    }

    @Test(expectedExceptions=RuntimeException.class)
    public void test2() {
        System.out.println("I am in test2 test method and it will fail.");
    }

    @Test
    public void test3() {
        throw new SkipException("Skipping the test3 test method!");
    }

    private int i=0;
    @Test(successPercentage=60, invocationCount=5)
    public void test4() {
        i++;
        System.out.println("test4 test method, invocation count: " + i);
        if (i == 1 || i == 2) {
            System.out.println("test4 failed!");
            Assert.assertEquals(i, 8);
        }
    }

}

Listener:

In listener class, I implemented ITestListener class and I explained its method.

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

/**
 * Created by ONUR on 11.12.2016.
 */
public class Listener extends BaseTest implements ITestListener {


    @Override
    public void onTestStart(ITestResult iTestResult) {
        System.out.println("I am in onTestStart method " +  getTestMethodName(iTestResult) + " start");
    }

    @Override
    public void onTestSuccess(ITestResult iTestResult) {
        System.out.println("I am in onTestSuccess method " +  getTestMethodName(iTestResult) + " succeed");
    }

    @Override
    public void onTestFailure(ITestResult iTestResult) {
        System.out.println("I am in onTestFailure method " +  getTestMethodName(iTestResult) + " failed");
    }

    @Override
    public void onTestSkipped(ITestResult iTestResult) {
        System.out.println("I am in onTestSkipped method "+  getTestMethodName(iTestResult) + " skipped");
    }

    @Override
    public void onTestFailedButWithinSuccessPercentage(ITestResult iTestResult) {
        System.out.println("Test failed but it is in defined success ratio " + getTestMethodName(iTestResult));
    }

    @Override
    public void onStart(ITestContext iTestContext) {
        System.out.println("I am in onStart method " + iTestContext.getName());
    }

    @Override
    public void onFinish(ITestContext iTestContext) {
        System.out.println("I am in onFinish method " + iTestContext.getName());
    }

    private static String getTestMethodName(ITestResult iTestResult) {
        return iTestResult.getMethod().getConstructorOrMethod().getName();
    }

}

TestNG.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNG Listener Example">
    <listeners>
        <listener class-name="Listener" />
    </listeners>

    <test name="TestNG Sample Test" preserve-order="true">
        <classes>
            <class name="SampleTest">
<!--                <methods>
                    <include name="test1"/>
                </methods>-->
            </class>
        </classes>
    </test>

</suite>

Pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>TestNGListener</groupId>
    <artifactId>TestNGListener</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.14.3</version>
        <scope>test</scope>
    </dependency>
    </dependencies>

</project>

Test Output:

[fusion_builder_container hundred_percent="yes" overflow="visible"][fusion_builder_row][fusion_builder_column type="1_1" background_position="left top" background_color="" border_size="" border_color="" border_style="solid" spacing="yes" background_image="" background_repeat="no-repeat" padding="" margin_top="0px" margin_bottom="0px" class="" id="" animation_type="" animation_speed="0.3" animation_direction="left" hide_on_mobile="no" center_content="no" min_height="none"][TestNG] Running:
I am in onStart method TestNG Sample Test
I am in Before Method! Test is starting!
I am in onTestStart method test1 start
I am in test1 test method and it will pass.
I am in onTestSuccess method test1 succeed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test2 start
I am in test2 test method and it will fail.
I am in onTestFailure method test2 failed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test3 start
Test ignored.
I am in onTestSkipped method test3 skipped
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 1
test4 failed!
Test failed but it is in defined success ratio test4
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 2
test4 failed!
Test failed but it is in defined success ratio test4
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 3
I am in onTestSuccess method test4 succeed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 4
I am in onTestSuccess method test4 succeed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 5
I am in onTestSuccess method test4 succeed
I am in After Method! Test is ending!
I am in onFinish method TestNG Sample Test
===============================================
TestNG Listener Example
Total tests run: 8, Failures: 3, Skips: 1
===============================================

Test Code:

https://github.com/swtestacademy/TestNGListener

References:

https://examples.javacodegeeks.com/enterprise-java/testng/testng-listeners-example/

http://www.guru99.com/listeners-selenium-webdriver.html

http://testng.org/doc/documentation-main.html#testng-listeners

http://learn-automation.com/what-is-testng-listener-and-how-to-implement-in-selenium/

http://toolsqa.com/selenium-webdriver/testng – listeners/[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]

Onur Baskirt

Onur Baskirt is a Software Engineering Leader with international experience in world-class companies. Now, he is a Software Engineering Lead at Emirates Airlines in Dubai.