Posted by mason
on June 29, 2006 at 1:48 AM PDT
Two friends of mine, both excellent Java programmers, just recently got into an incredibly heated argument on the subject of unit testing. On the left was the server-side web application developer and on the right was the client-side Swing developer. On the one side a happy village, with crops of green test lights in neat little rows, on the other a dark jungle where code roamed free and wild, and any step you took could be your last.
The argument started with the web developers assertion that unit tests could test just about everything. And to look at her code, you'd almost feel the same way. As a TDD proponent, she had nearly twice as many tests as she has methods and classes. Not a single like was untested... Well, that probably isn't 100% true, but it damn near looked like it.
What she did not have, however, was a bug (of any sort) that lived longer than ten minutes, and even those were rare beasts. Sure, there were bugs in the overall system, where her code touched others in the organization, but not in hers. Every so often, entropy would take hold, and a requirement would be misunderstood, or a test that wasn't testing properly because of a misplaced semicolon or negation, and yes a bug would show it's ugly head. But it would be squashed with a new test that legislated it out of existence, never to return. So when you look at her development process, you could see that her testing advocacy was not unfounded jibber jabber... It was as solid as a rock, a fortress of evidence at which all arguments to the contrary would simply fail. But most importantly, it was a thorn in the side of the Swing developer.
He was a very good programmer. He had to be, because he was not just a swing developer, but a cross platform swing developer. That meant he not only had to deal with the nightmare of caveats, disclaimers and javadoc half-truths that is Swing on one platform, he had to find a way to deal with that on three platforms! So bugs were a common experience for him. They could be reduced in scope and danger, but they could not be eliminated. Sure he could write a unit test here and there, for the code that calculated the size of a box, or that that checked the web service for the login information... but when it came to the GUI itself, tests were simply impossible.
Impossible!? She would yell at him, how could it be impossible? It's all just inputs and outputs, right? But is it as simple as that? Logically it must be, because that is all a computer is... yet try as he might, he steadfastly refused to believe Swing was testable. Their argument was going on for days, and so I decided to look into the problem and see what I could see...
From my point of view, a swing GUI and a web GUI were essentially, from a fundamental level, really the same thing kind of thing. They displayed widgets that you could click on and type on and then things happen and the UI changes in some way. Different implementation, but same general purpose. So why were web-based GUIs so simple to test (with HTMLUnit, HTTPUnit, etc), but Swing-based GUI's so impossible?
Read on into the extended section to see why I discovered that, yes, Swing code is fairly untestable, almost by design... but that doesn't mean it's an inherant flaw in GUIs... there may just be hope after all in the next generation of GUI toolkits.Fundamental Differences
Well, if you notice the two tools that I mentioned above in terms of test frameworks for the web, you'll see the problem right away, well, the FIRST problem... Those frameworks don't test the rendering of the web GUI, they just test the layout definition or lower level interface protocol (respectively) that is used to create/talk to the GUI. (It is simply assumed that the rendering will not fail at the widget level. Of course, they will want to use div version A for windows and div version B for mac, and they can test that is working, but they never test the actual rendering itself... That is the browser's problem.) For Swing, there is no data transport layer, beyond the interal magic of calling a method, between the drawing of the UI and the underlying business logic of the widgets themselves. Oh sure, there is an attempt at an MVC architecture with many of the JComponents, which makes components a little less messy, but it's not the same clean, hard presentation/computation break that a web GUI gets.
Can Swing components be split into pure UI, pure business logic and a decoupled interface in between?
So, I thought, how hard would it be to teach Swing to fake it? Take a few of the components and split them into a layout definition, a UI renderer and a logic engine, all separated by something like a nice RMI interface. Basically I tried a quick test to see if I could decouple the abstract â€œwidgetâ€ logic from the actual drawing of the widget itself, and make Swing work a bit like X11. I'm a fairly good engineer, I should be at least able to make a small dent, eh?
Ah, no... This turns out to essentially be impossible. Too many of the swing classes use twisted package/private inner classes to do their real work. I even tried copy and pasting the Swing source code required for one component into a new set of packages, but it after about 30 classes worth of source, I was beginning to see that any single Swing component was intertwined into so many other bits and pieces that by the time you pulled it â€œfreeâ€ it will have taken the entire javax source tree with it.
In the time that I would spend walking through the complicated intricacies of a combo box, or (god forbid!), a JTree, I could probably make some fairly decent headway on an entirely new widget system... (And no, that last line is not hyperbole... Just for kicks, I took a JOGL canvans and implemented crude buttons, panels, scroll bars and tables. It took a day or two, and I am no UI designer, so they looked like things that Apple folks have nightmares about, but it was proven possible to do, and it worked fairly well for the purposes of the test. My point being, the internal complexity of swing, justified or not, is too complex for it's own good)
Worst of all, there is no real uniformity in all the components, meaning, even if you could split the UI and the business logic with a solid interface for ONE component, you would have to do all the work again from scratch for the next, so little of the identical code between them is shared. And even after that, building a generic enough communication protocol (forget RMI, that is for sure!) on all that inconsistency would be remarkably difficult. The ComboBox and the List and the Table and the Spinner and the Slider and the Tree all have similarities... but only at the name level, not at the actual compiled class level. They all have things called Model and Renderer, and yes, sometimes more than one component can use bits or pieces from another, but when you spend a lot of time with them, you notice that they are often more incompatible than not. Internally, the Document rendered by the JEditorPane is just a fancy tree... yet it can't be rendered by the JTree. There are three different models (ListModel, ComboboxModel, SpinnerModel) for what amounts to the same underlying data structure. Some components are triggered via an ActionListener, and some though an ItemListener, or maybe a ListSelectionListener, and others through both. Some of those listeners fire two events, one for â€œdoingâ€, and one for â€œdoneâ€, while others just fire the â€œdoneâ€. These inconstancies mean that any interface between the two worlds is going to be unnecessarily complex and full of disclaimers and â€œyes butâ€s. (It also means even you normal Swing code is going to be far more complex than it â€œneedsâ€ to be to get the job done.)
Through much trial and effort, I was never quite able to separate the visual UI from the pure business logic. It's surprising this is the case, because Swing is based on an MVC model. The problem is quite simply that the View and the Model of Swing's version MVC must talk to each other constantly, yet indirectly, and the â€œcontrollerâ€ is almost universally untouchable... Except, of course, in the cases when it's embedded in the View by default. The model DOES manipulate the view through â€œsort ofâ€ asynchronous events (well, it's manipulated through listeners that could theoretically be implemented asynchronously), but the view drives data access directly through method calls and expects results pronto before it will move on. Manipulating the view through the controller MUST be done on the AWT event thread to avoid random exceptions, and, except in the cases of an embedded controller, there is no framework for manipulating the model unless you write it yourself. And in the end, you'll tend to intertwine most everything anyway, at least at some level, since that is the only reasonable way to properly keep events in line with widget state and keep from having to subclass every single Swing component you want to use.
Can Swing's event-driven model be tamed?
The second major problem to testing Swing is completely different event model between Swing and the web. The paradigm of client-side GUI based programming is that it is driven by events. At any given moment, the â€œstateâ€ of a Swing application is in flux. Events can be triggered in any order and at any time, and the number and frequency of the events are at the whim of the user. While it's true that a servlet could be considered â€œevent drivenâ€ as well, it is only driven by a single event at a time, and the state of the GUI is static between events (discounting AJAX, which is discussed later): one event comes in, with various properties and values attached to it, and one action is taken.
You can mimic this with Swing, but only half way, and with poor results. You can have something collect up events as they happen, and then at a predetermined time, push them all to some engine that decides what do to, but that is inefficient, and most importantly, very non-responsive. Instead you can pass the events â€œas they happenâ€, but instead of a bunch of tiny listeners taking action when they feel like it, pass events directly to a single handler (or set of handlers) that processes events in a testable way.
This is essentially Swing's â€œActionâ€ model (except the one implemented in Swing only understands one type of event: the â€œactionâ€ event, which means different things to different components). It sounds like a good plan until you try it on anything complicated... You can collect and dispatch events just fine, and your fully tested handlers (your Action) will correctly processes the event and perform the proper actions, and the handler will be reusable for all widgets that perform the same action, but what happens when the event handling is complete and the state of a widget needs to be updated? The Action model is limited in that it really only understands enabled/disabled, and a perhaps some undocumented properties. This model is too far away from the GUI widgets (at best, everything is buried under layers of JPanels and other containers, assuming you can even get a reference to the root pane to start your search), and swing has no simple way to query for them.... Sure, for example, you received a double click at x,y, but where is that, what is it touching? Or, yes, the table cell was edited, but the table is buried under a JScrollPane that is hidden behind a JTabbled pane, which is in a dialog box that you popped up, and the only way to get to it is to keep a static reference to it somewhere or iterate through lists of lists of unnamed components.
Ok, how about the same â€œActionâ€ model, but you write your own event system such that every event is paired with a reference to a Swing widget along side the event that it triggered? Better, maybe, but with one huge disadvantage... Swing widgets extend from strange things sometimes, and none of them implement any useful interfaces for switching off and on common fields (no â€œSelectableâ€ interface, etc). Sure everything is a JComponent, but that doesn't give you a lot to work with. You have to cast them at some point, or use reflection, to change the various values. Casting to various hard classes fails when you want to do JUnit testing because you don't want to be forced to use Swing components in your tests...
So, simple enough, you make a lot of interfaces for the â€œcommonâ€ bits, subclass all the Swing classes implementing your own interfaces, and now, finally, you can do some JUnit style testing that will test the business logic and the UI logic together, without needing to invoke Swing directly. You're stretching the framework pretty far to do it, however, and your code will end up being filled with switches and if/thens while you try to figure out what kind of object you just got so you can call setSelected() on it.
So what did all that effort eventually solve again?
So, the lesson of my Swing slashing was this: Unless you are willing to litter your code with static variables holding bits of widgets, or attempt to subclass every swing widget with your own custom interfaces, or fill your code with ugly spaghetti code constructs, you are just better off writing the UI and the business code in whatever way Swing makes the least complicated, use the proper MVC bits when they are available, of course, compartmentalize as much as possible in the Actions and various subclasses of listeners, ignore the MVC violations that happen, and mostly give up on thorough unit testing.
Swing was simply not created with JUnit style testing in mind, and you won't find it easy to force into submission.
So, is that then end of the story?
Not quite yet...
AJAX -- I mentioned it was coming up eventually -- is very soon going to make web-based GUIs as complex as anything in Swing's arsenal. As that happens, the testability of the web-based GUI drops considerably. There is still a strong, decoupled interface between the UI and the rest of the application, but the UI is taking on more and more responsibility, and it's acting in ways that are invisible to the current web-based testing frameworks. Soon web developers like my TDD loving friend, will be in the same boat as Swing developers. Both sides will be rightly bemoaning the neigh impossibility of testing properly against their UIs.
However, just because the current crop of UI toolkits are difficult to test, that doesn't mean that GUIs in general, or GUIs as a paradigm, are doomed to untestability. From all of this failed experimentation, I think I just glimpsed a light at the end of the tunnel. There is a way to get the same functional and unit test coverage from a responsive, client-side GUI as you can get from servlets and html... but you have to just use the right kind of GUI toolkit... One that I am not quite sure if it exists yet, or at least, one that may just be in it's infancy.
A (possibly) new GUI paradigm will be the subject of my next installment...