Skip to main content

Swing Automated Test Harness (SwATH)

2 replies [Last post]
kolotyluk
Offline
Joined: 2007-01-02
Points: 0

I have an idea for a JSR, but I’m not currently a member, nor is my current employer Kodak a member. I was wondering who I might contact as a sounding board to see if the idea is worth pursuing – without having to write a formal JSR Proposal.

One of the difficult areas of software testing is Graphical User Interfaces. It is particularly hard to automate such testing because of the dynamic and interactive nature of the GUI.

There are a number of tools on the market to aid in automated GUI testing but these usually require a huge investment in design, implementation, and (well) testing, and are often not very flexible especially in regression testing when the UI changes.

I have some experience in automated testing in wireless networking. Automating testing was very practical and straightforward because usually you could create a test harness that simply monitored and replied to network traffic and protocol simulation.

For some time now I have desired something similar for GUI testing. What I think would work well is to embed a generic test harness into the Swing API. The test harness would consist of a specified TCP port to communication with the test system. For example run your program with something like

-Xswath=123.123.123.123:1234

All Swing activities would be translated into a series of well defined XML messages. For example

</p>
<p>      . . .</p>
<p>

This is a very crude example. The point is:

This XML would be emitted every time there is a change in the UI, with little or no special effort on the part of the programmer. For example the ids might have to be set by the programmer, but the rest would be automatically generated similar to serialization. This XML would be easily parsed by the automated test system using a variety of technologies.

In turn the test system could respond with actions, for example

</p>
<p>

One important feature of such a system is that over time the UI could change, the button might move to a different location, the text on the button could be in a different language and font, an icon could be added to the button, but the button’s id remains the same. This makes regression testing easier.

I feel there would be a great deal of utility in supporting such a feature in Swing. Through automated testing of the GUI, the cost of testing could be reduced dramatically, and the quality of software could be assured more affordably.

There are a great many other issues and related opportunities for specification and standardization. For example, tools to help build the XML recognition engine in the test system; integration with JUnit and other technologies; IDE integration and plugins; etc.

I would greatly appreciate hearing from someone on whether this idea/proposal has merit.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
aberrant
Offline
Joined: 2006-02-02
Points: 0

This functionality might be interesting to implement as an MXBean. Instead of listening on some custom port.

http://java.sun.com/developer/technicalArticles/J2SE/mxbeans/

coxcu
Offline
Joined: 2003-06-11
Points: 0

I suggest you repost here:

Java GUI Testing Group
http://tech.groups.yahoo.com/group/java-gui-testing/

I doubt a JSR would be approved for this. Non-GUI (JUnit, TestNG, etc...) unit testing is much better understood and well established, yet has no official Java presence via a JSR.

Abbot is probably the leading open-source tool for Java GUI testing. That would be a good place to start.
http://abbot.sourceforge.net/doc/overview.shtml

The approach you are suggesting might ultimately require a custom Toolkit/GraphicsEnvironment. In theory, those are pluggable although implementation is quite a bit of work. I vaguely remember some company implementing a custom Toolkit/GraphicsEnvironment for GUI testing, but can't find a link. Fortunately, you can do a lot by adding a Toolkit listener.

Toolkit.addAWTEventListener()
http://java.sun.com/j2se/1.4.2/docs/api/java/awt/Toolkit.html#addAWTEventListener(java.awt.event.AWTEventListener,%20long)

One approach to Java GUI testing is outline here:

"XP Testing a GUI with FIT, Fitnesse, and Abbot"
http://www.csis.pace.edu/~bergin/xp/guitesting.html