Posted by aditya_dada
on July 12, 2006 at 1:57 PM PDT
My recent presentation on 'Smoke Tests to Signal Test Readiness' at the Better Software Conference, and a review of a few talks that stood out.
I recently presented at the Better Software conference in Las Vegas. It was my first time at this conference and I had a great time attending, and presenting there.
I was scheduled to present on the 29th and I arrived at the conference on 28th at lunchtime (now that's one way to start a conference!).
My presentation on 'Smoke Tests to Signal Test Readiness ' went off well, with the room quite full of audience. I started my talk by covering typical problems that face complex software complex stacks. I then discussed definitions and purpose of smoke tests, different types of release models (Tinderbox, nightly weekly, milestone), different types of smoke tests to find bugs for the different release models. Features for designing smoke tests (coverage, speed, reporting, management, tools) and challenges in designing proper test suites followed, namely:
Â· Complex Software Stack
Â· Multiple developers
Â· Tests for different teams and purposes
Â· Test enhancement Strategy
After describing a process to create smoke-tests, the talk covered a real-world example from our GlassFish (and Sun Java System Application Server ) testbase. I ended my talk with benefits & recap followed by Questions & Answers session. The audience raised very relevant questions at the end that made for some great discussions.
Even though only a small percentage of my audience had used GlassFish or Sun Java System Application Server , I handed out some business cards on GlassFish and the Quality Portal , as well as CDs and even a few T-shirts, and the response was very positive. I also helped educate some members of the audience that Sun Java System Application Server was free to download, and that GlassFish was an open-source project. Again the response was very positive, and I hope some of them contribute to the GlassFish Project in the near future.
There were many interesting topics people presented, however three stood out for me.
One was a very technical and enjoyable talk titled 'Software production line automation ' by C. Thomas Tyler . Since my own talk dealt with automation in some ways, it was interesting to me that his talk had a lot of ideas very similar to those in mine, and to those that we follow at Sun. It was good to see that others faced similar issues as we did. The presentation focused on trying to bring the line automation into software, by parallel-izing software development, and improving automation with the use of tools. In addition, the presentation covered ways to incorporate Agile methodology and create a system where a machines could create products that maybe shipped with the current state of features, where machines can provide automated answers to questions like - "which bug fixes went into which build"?
Another very entertaining and educational talk, was given by Payson Hall on 'How to Estimate Anything '. He claimed in his talk that if we (the audience) were to follow his techniques, it would guarantee an improvement in our estimation by 20%, or he would come to where ever we were in the world, buy us lunch, and discuss what could be done better. Now that's a claim I'd like to challenge :) Payson had two demonstrations in his talk. One was to estimate lengths of different ropes - once wrapped neatly, held in hand, and shown slowly and quietly to the audience, then all untangled with Payson running wildly around the audience, then finally, with me as a baseline to measure it against. The audience 'estimated' the lengths. The results:
Â· Estimates for shorter ropes were more accurate.
Â· Estimates made when compared to a baseline were more accurate.
Â· Estimates made when the target was moving, and a lot of noise interfered - were less accurate.
Another demonstration had to do with estimating the number of credit cards in the room. Same question, asked three times, with different amounts of time allowed to answer them yielded three different answers. Interestingly, people's observations related in some way to their cultural backgrounds! One thing to note was that the art of estimation, as old as it maybe, still poses challenges in almost every organization for almost every project.
Joel Splosky gave a keynote presentation on 'Donâ€™t Settle For Better Softwareâ€”Make Truly Great Software '. He brought out 3 points that he believes makes the difference between a good product, and a market leader - knowing what the users want, keeping the emotional aspect in mind and obsessing over aesthetics (iPod vs Creative's Zen). It was one of the most entertaining presentations at the conference. In a simple way, Joel brought out a subtle idea - not all products with good functionality become market leaders.