SIMPLIUM-Simple test framework for Selenium.

Overview

Simplium aids the developer to create selenium based unit test that can be executed in different browsers and environment. With environment means the same test can be run against the developers own machine as well as a test machine somewhere on the network or even the production machine.

A Simplium unit tests doesn't have any reference to which URL the test is executed against as well as no reference to which browser the test should be executed in. Instead the Simplium test cases are annotated with Simplium specific annotations which allow Simplium to take control of the test cases when executed.

Simplium looks for which environment it is executed in (development, test or production) and based on this information Simplium will execute the test cases against different servers, against a local selenium remote control or selenium grid and execute each test in the browsers that the environment has specified.

Download:

Download the latest version of Simplium from http://simplium.sektor.se/download.html
Current version is 0.4.

Quick Example

Take a look at a ordinary Selenium test case.

import junit.framework.TestCase;

import com.thoughtworks.selenium.*;

public class ExampleSeleniumTest extends TestCase {

    private Selenium selenium;

    public void setUp() {

        selenium = new DefaultSelenium("localhost",

                4444, "*firefox", "http://www.google.com");

        selenium.start();    }

    public void testGoogleForSomeThings() throws InterruptedException {

        selenium.open("/");

        assertEquals("Google", selenium.getTitle());

    }

    public void tearDown() {

        browser.stop();

    }

}

This test case has the selenium server host and port specificed in the source code, as well as the browser to run the test case in and also which URL the test should run against.

What if we would like to run this test against our development machine and test machine as well? And what if we also would like to run this test in Internet Explorer, Safari and Chrome? It is here Simplium comes to the rescue.

A Simplium based Selenium test case can look like this

package com.company.test;

import junit.framework.Assert; 

import org.junit.Test;

import org.junit.runner.RunWith;

import org.simplium.SimpliumWebTest;

import org.simplium.junit.SimpliumJUnitRunner;

@MyCompanyWebTest

@RunWith(SimpliumJUnitRunner.class)

public class ASimpliumGoogleTest extends SimpliumWebTest {

        @Test
     public void testGoogleTitle() {

               selenium.open("/");

               Assert.assertEquals("Google", selenium.getTitle());        }}

The @MyCompanyWebTest is a meta annotation that contains a number of Simplium specific annotation. This annotation combine with a specific JUnit Runner the Simplium can in the backgrund determine the current enviroment, what browser to execute the test case in and to set up a Selenium server.

package com.company.test;

import java.lang.annotation.Documented;

import java.lang.annotation.Inherited;

import java.lang.annotation.Retention;

import java.lang.annotation.RetentionPolicy;

import org.simplium.annotation.Browsers;

import org.simplium.annotation.Enviroments;

import org.simplium.annotation.Screenshot;

import org.simplium.annotation.Screenshot.CaptureRule;

@Browsers

@Enviroments(

        development = {"localhost","4444","http://localhost:8080"},

        test = {"grid.company.com","4444","http://test.google.com" },

        production = {"grid.company.com","4444","http://www.google.com"}

)

@Retention(RetentionPolicy.RUNTIME)

@Documented

@Inherited

public @interface MyCompanyWebTest {

}

The @MyCompanyWebTest is listed abow. The @Browsers indicates what browser to use. For now the default browsers are used which means that each test case run in the development enviroment is run in Firefox and Internet Explorer. When using test it is run in a number of other browsers as well.

@Enviroment defines the Selenium server host, port and the URL to test against for each of the three specificed enviroments. This means that for @MyCompanyWebTest a local Selenium server is started in development enviroment and the test is executed against localhost:8080. When running in test Simplium lets a Selenium grid execute the test cases. The grid is located at grid.company.com:4444. The test cases are then executed against test.google.com. Prodution also uses the Selenium grid but the test cases are executed against www.google.com.

To take control of which browser that the test should be executed in and not use the default ones the @Browsers annotation can be feed with this information.

@Browsers(

        development = {"*iexplorer","*firefox"},

        test = {"*iexplorer","*firefox","*googlechrome","*opera","*safari"},

        production = {"*iexplorer","*firefox","*googlechrome","*opera","*safari"}

)

The @Browsers annotation can also be used on a test case method to override the @Browser annotation on the class level. This if some part of the web application can't be executed with certain browser(s).

Simplium also aids the developer to find out why a test case fails. When Simplium javacode that a test case fails, through a assert or by a unknown not caught exception Simplium can be instructed to take a screenshot of the browser so that developer can see exactly what happened on the screen when the fail occured. This is done by marking the meta annotation with the @Screenshot annotation.

@Screenshot(CaptureRule.CAPTURE_WHEN_EXCEPTION_OCCURES)

public @interface MyCompanyWebTest {

}

Simplium will create a .png under the tmp directory followed by the current execution time and the test class that failed.

There is a number of other annotation that also can be used, @Delay, @RunOnlyInDevelopment, @RunOnlyInTest and @RunOnlyInProduction.

How to setup a simple Simplium testcase

Step 1 - Create annotation

First we need to create an annotation that all our Simplium test cases must be annotated with.

package com.company.test; 

import java.lang.annotation.Documented;

import java.lang.annotation.Inherited;

@Documented

@Inherited

public @interface MyCompanyWebTest {

}

Note that we annotate this annotation with other annotation making this a so called meta annotation. The @Inherited makes sure that this annotation is inherited for all sub classes to a MyCompanyWebTest annotated super class. @Documented makes this annotation show up in JavaDoc.

Step 2 - Define the enviroments

The newly created annotation must define to Simplium the test enviroments for development, test and production.

package com.company.test;

import java.lang.annotation.Documented;

import java.lang.annotation.Inherited;

import org.simplium.annotation.Enviroments;

@Enviroments(

        development = {"localhost","4444","http://localhost:8080"},

        test = {"grid.selenium.com","4444","http://testenviroment.com" },

        production = {"grid.selenium.com","4444","http://www.company.com"}

)

@Documented

@Inherited

@Retention(RetentionPolicy.RUNTIME)

public @interface MyCompanyWebTest {

}

Each enviroment takes three parameters, the host name of where the selenium server is located, the port the selenium server is listing on and where the selenium test case should be executed against.

In development if there doesn't exist an selenium server on the specified host and port a server is started in the background. For test and production the selenium grid should be used.

The base url where the selenium test case is executed should in the development mode be set to the developers machine, localhost, combined with the port for the local test enviroment. The base url for test and production should point on some test enviroment and the production URL. When running in test or production enviroment these URL can be changed with a system property.

Step 3 - Define the browsers

So we have definied the enviroment but not which browsers that the test cases should be run in.

package com.company.test;

import java.lang.annotation.Documented;

import java.lang.annotation.Inherited;

import org.simplium.annotation.Enviroments;

@Enviroments(

        development = {"localhost","4444","http://localhost:8080"},

        test = {"grid.selenium.com","4444","http://testenviroment.com"},

        production = {"grid.selenium.com","4444","http://www.company.com" }

)

@Browsers(

        development = { "*iexplore"   "*firefox"      },

        test = {

               "*firefox on Windows"

               "*firefox on MacOS"

               "*firefox on Linux"

               "*iexplore on Windows"

               "*iexplore on MacOS"

               "*safari on MacOS"

               "*safari on Windows"

               "*opera on Windows"

               "*opera on MacOS"

               "*opera on Linux"

               "*googlechrome on Windows"

        },

        production = {

               "as test"

        }

)

@Documented

@Inherited

@Retention(RetentionPolicy.RUNTIME)

public @interface MyCompanyWebTest {

}

With the use of the @Browser annotation we define that in the development enviroment the test case should be executed in iexplore and firefox. For the test enviroment the same test case is run in a number of browser on different plattforms. Here we use Selenium grid to distribute the execution of the test case to machines running the different plattforms and having the different browsers installed. So the machine that starts the test cases doesn't need to have any browsers installed. For production we simply use the same set as for the test enviroment.

Step 4 - Create your Simplium test case

Now we are ready to start developing the test case.

package com.company.test;

import junit.framework.Assert;

 import org.junit.Test;

import org.junit.runner.RunWith;

import org.simplium.SimpliumWebTest;

import org.simplium.junit.SimpliumJUnitRunner;

@MyCompanyWebTest

@RunWith(SimpliumJUnitRunner.class)

public class ASeleniumTest extends SimpliumWebTest {

        @Test

        public void myTestMethod() {

               selenium.open("/mywebapp");

               Assert.assertEquals("Example", selenium.getTitle()); }}

The test case must be annotated with the created annotation @MyCompanyWebTest. To run the test case with JUnit the @RunWith annotation should be presented and the SimpliumJunitRunner class must be used. Also the test case must extend the SimpliumWebTest.

Note that the test case doesn't have any information about the executing enviroment such as where the selenium server is located or which browser the test should be executed in.

Step 5 - Run in development mode

When running the test case in any modern IDE (here in Eclipse) Simplium execute the test cases in the different browsers specified by the @Browsers annotation.

Note that the test case is run twice, on for each browser definied in the development enviroment.

Step 6 - Run in test mode

So now the developer has tested the application on his developer machine but what about running the test case again but now against a test machine? Simplium makes this simple. Just add the VM argument -Denviroment=test when running the test cases and Simplium will automaticly use the Selenium grid definied under test and execute the test cases with the browsers that has been defined under test.

Step 7 - Run in production mode

The same applies for running in production mode. Use the VM argument -Denviroment=production to make Simplium execute the test cases with the production settings.

Step 8 - The rest

There is a number of annotation that hasn't been mention so far so to get familiar with these this section explain the usage.

@Screenshot

Simplium also aids the developer to find out why a test case fails. When Simplium javacode that a test case fails, through a assert or by a unknown not caught exception Simplium can be instructed to take a screenshot of the browser so that developer can see exactly what happened on the screen when the fail occured. This is done by marking the meta annotation with the @Screenshot annotation.

@Screenshot(CaptureRule.CAPTURE_WHEN_EXCEPTION_OCCURES)

public @interface MyCompanyWebTest {

 }

@Delay

@Delay marks a class or method that all selenium API calls should be delayed. This is usable when debugging a test case. The defaul delay is 1000ms (1s) and this can be changed by setting the delay with the annotation, @Delay(2000) gives 2 second delay.

package com.company.test;

 import junit.framework.Assert;

 import org.junit.Test;

import org.junit.runner.RunWith;

import org.simplium.SimpliumWebTest;

import org.simplium.junit.SimpliumJUnitRunner;

 @MyCompanyWebTest

@RunWith(SimpliumJUnitRunner.class)

public class ASeleniumTest extends SimpliumWebTest {

    @Test

   @Delay

   public void searchOnGoogleForGoogle() {

      selenium.open("/");

      selenium.type("q", "google");

      selenium.click("btnG");  }}

@RunOnlyIn*

There are three annotations @RunOnlyInDevelopment, @RunOnlyInTest and @RunOnlyInProduction that can be used to mark classes or method so that it is only run in the specified enviroment. In below example the test case searchOnGoogleForGoogle will only be run under the developmen enviroment. When run in test or production this test case will be ignored.

package com.company.test;

 import junit.framework.Assert;

 import org.junit.Test;

import org.junit.runner.RunWith;

import org.simplium.SimpliumWebTest;

import org.simplium.junit.SimpliumJUnitRunner;

 @MyCompanyWebTest

@RunWith(SimpliumJUnitRunner.class)

public class ASeleniumTest extends SimpliumWebTest {

    @Test

   @RunOnlyInTest

   public void searchOnGoogleForGoogle() {

      selenium.open("/");

      selenium.type("q", "google");

      selenium.click("btnG");

   }

}

@Timeout

Annotation that can be used as a meta annotation, class annotation or method annotation to indicate the timeout time Selenium will use for its "open" and the "waitFor*" actions. It is specified in milliseconds.

package com.company.test;

 import junit.framework.Assert;

 import org.junit.Test;

import org.junit.runner.RunWith;

import org.simplium.SimpliumWebTest;

import org.simplium.junit.SimpliumJUnitRunner;

@MyCompanyWebTest

@RunWith(SimpliumJUnitRunner.class)

public class ASeleniumTest extends SimpliumWebTest {

    @Test

   @Timeout(60000)

   public void searchOnGoogleForGoogle() {

      selenium.open("/");

      selenium.type("q", "google");

      selenium.click("btnG");

   }

}