Marathon Tutorial – v0.13

Preface

Marathon is a tool that helps writing user acceptance tests. This tutorial will cover different aspects of development assisted by tests.

W're going to start with an example of writing user acceptance tests against an application already developed, the calculator application.

In the “tutorial” directory you will find one subdirectory per step.

This version of the tutorial is based on the 0.13 build of Marathon you can download from : http://marathonman.sourceforge.net/download. The tutorial was written on a Windows system (tested on Win98 and Windows 2000).

Calculator example.

A calculator application was developed and the result is bundled in a jar file (tutorial/lib/calculator.jar). Launch it just by double-clicking¹ the jar file. Play with the calculator, next figure shows the application:

Step 1: Write some acceptance tests

We want to write tests for the calculator. We expect the calculator to be able to:

1. add two values

2. subtract two values

3. clear the current value

Let's write a test script to test addition, start the Marathon IDE (using the start-IDE.bat file):

1. create a new empty test (a test is a Python script file)

   • right click on the current directory (in the left-side tree view)

   • choose the “New Test From Template” entry

   • you'll be prompted for the the name (enter testAdd), a file named testAdd.py is created in the directory

   • Then you'll be prompted to select the script template file (a template file name ends with the mtf extension), select step1/testCalculator.mtf

2. write the test

   • goto the line containg the Python pass keyword

   • remove it

   • click on the record button in the toolbar (the red circle)

   • enter "12 + 4 =", the result is displayed (16), if you look at the Marathon IDE you can now see:

                       window('Calculator Example') 
                       click('1')
                       click('2') 
                       click('+') 
                       click('4') 
                       click('=')
                       close() 
                       

   • right-click (on MS-Window systems) in the the text area, to open the contextual menu,
     
     
     

     
     
     select assert text='16'

   • stop recording (the blue rectangle button), the script now looks like:

                       window('Calculator Example') 
                       click('1')
                       click('2') 
                       click('+') 
                       click('4') 
                       click('=')
                       assertText('JTextField', '16')
                       close() 
                       

   • save the script²
     
     
     

3. replay the test (click on the green arrow). Change the asserted value to some bad value and see the error report

4. you can create two more tests: testSubtract and testClear³

                       window('Calculator Example') 
                       click('1')
                       click('2') 
                       click('-') 
                       click('4') 
                       click('=')
                       assertText('JTextField', '8')
                       close() 
                       

   and

                       window('Calculator Example') 
                       click('1')
                       click('2') 
                       click('-') 
                       click('4') 
                       click('=')
                       click('C')
                       assertText('JTextField', '0')
                       close() 
                       

Step 2: Refactor the Tests

As the Marathon acceptance tests are written in Python scripting language it helps managing the tests as you would do in any programming development: make maintenance easier and reuse of code.

Can we refactor anything in the three test scripts? Let's try. What can we refactor? What don't we like right now?

1. Writing one more test, without using the recording feature of Marathon, is hard.

2. Reading the tests to undrestand what is going on is not easy.

3. Currently the tests is saying: click on the component named '1' (for instance), if we later want to change the name of our component we will have to modify all our scripts

4. All the scripts start and end with the same sequence...

Wouldn't be easier (or nicer) to write:

                    enter('12')
                    add('4') 
                    assertResult('16')
                    close()
instead of
                    window('Calculator Example') 
                    click('1')
                    click('2') 
                    click('+') 
                    click('4') 
                    click('=')
                    assertText('JTextField', '16')
                    close() 

We need to create some helper methods: enter, add, subtract, clear, assertResult and assertCleared. As we want them to be used by all the scripts, we're going to create a new Python module:

tutorial/step2/util.py

from marathon.playback import * from defaultFixture import * def enter(value): window('Calculator Example') enterValue(value) def add(value): click('+') enterValue(value) def subtract(value): click('-') enterValue(value) def clear(): click('C') def assertResult(expected): click('=') assertText('JTextField', expected) def assertCleared(): assertText('JTextField', '0') def enterValue(value): for char in value: click(char)

Here is some explanation about this script:

• The import statements say we are using code from the named modules (marathon.playback and defaultFixture).

• The enter method first waits for the calculator main window to be displayed, next calls the enters the value.

• The add method first simulates a click on the plus button and the enters the value.

• The subtract method first simulates a click on the minus button and the enters the value.

• The clear method simulates a click on the clear button.

• The assertResult method sumulates a click on the equal button and checks if the current result is equal to the expected value.

• The assertCleared method checks if the current result is equal to the current value is zero (meaning cleared)

• The enterValue method simulates clicks for all the caracters of the value

Now, we can rewrite the three testing scripts as :

from step2.util import *

def test():
    enter('12')
    add('4')
    assertResult('16')
    close()

from step2.util import *

def test():
    enter('12')
    subtract('4')
    assertResult('8')
    close()

from step2.util import *

def test():
    enter('12')
    subtract('4')
    assertResult('8')
    clear()
    assertCleared()
    close()

Step 3: More Tests...

Now we can write more tests, without even recording user actions.

from step3.util import *

def test():
    enter('12')
    add('4')
    assertResult('16')
    add('30')
    assertResult('46')
    close()

from step3.util import *

def test():
    enter('12')
    add('4')
    assertResult('16')
    subtract('5')
    assertResult('11')
    close()

Now, you can launch the Marathon IDE again, load the new scripts and run them.

Step 4: Automate tests with JUnit

At this stage if we want to run the tests we must start the Marathon IDE and manually open (load) each test script and run it.

Marathon smoothly integrates with JUnit (a regression testing framework), so all the tests scripts can automatically be run à la JUnit. Two helper classes, MarathonTestSuite and MarathonTestCase, are provided in the net.sourceforge.marathon.junit package you can use to start the tests scripts, written in Pyhon, from Java.

Let's write a test suite that uses the scripts written so far⁴.

TutorialTestSuite.java

package step4; import junit.framework.Test; import junit.framework.TestSuite; import net.sourceforge.marathon.junit.MarathonTestCase; import java.io.File; import java.io.FileFilter; import java.net.MalformedURLException; public class TutorialTestSuite { public static Test suite() { TestSuite suite = new TestSuite("Tutorial Acceptance Tests"); addTests(suite, "step2"); addTests(suite, "step3"); return suite; } private static void addTests(TestSuite suite, String dir) { File file = new File(dir); File[] files = file.listFiles(new TestScriptFilter()); for (int i = 0; i < files.length; i++) { try { suite.addTest(new MarathonTestCase(files[i].toURL())); } catch(MalformedURLException me) { me.printStackTrace(); // the system has some problem here } } } static class TestScriptFilter implements FileFilter { public boolean accept(File pathname) { return pathname.getName().startsWith("test") && pathname.getName().endsWith(".py"); } } }

After compiling this file, in the class directory, we can run the tests using the start-JUnit.bat bat file.

You can download the tutorial here and the.