Managing the Testing Process

Only honest data yields worthwhile information.

One development engineer told me that he “felt very depressed” after reading the risk analysis documents my test staff had prepared. I find it more depressing, however, to ignore the possibility of failure during development, only to live with the reality of preventable failure after release.

The most dangerous kind of wrong is the kind of wrong that sounds reasonable, as I've said elsewhere in this book.

People, regardless of where they are, are not widgets. They want responsibility. They want respect. They want to be valued. They want us to be interested in developing their skills and their careers, even if they work for a third-party contractor. They want feedback.

I sometimes hire people whom I think will grow into a job, but I never hire people I think are trying to con their way into a job.

The non sequitur interview involves asking some question that seems to be analogous or related to testing. The classic example is, “Tell me how you would test a salt shaker.” I find this silly. I call this a non sequitur interview style because it does not follow that, just because someone can craft a clever tale about testing simple real-world objects, they can test complex software and systems. You are hiring a test professional, not a raconteur.

You also might want to think twice about certain types of metrics—in particular, those that address questions such as “Who found the most bugs overall?”, “Who spent the most time testing?”, and “Who found the most bugs per hour spent?” I find these numbers dangerous, and I don't generate them. As soon as you begin analyzing employee performance data, you take risks. If people know you are measuring them this way, they will soon figure out how to manipulate the numbers.

Skip. You've decided to skip the test for this pass, typically because it's a relatively low priority. Again, the Comments column of the test case summary worksheet should reflect why the test is not run. One way to think of the Skip state is “I could have run the test, but I didn't want to.”

1. Technology Trigger. The first phase of a Hype Cycle is the “technology trigger” or breakthrough, product launch, or other event that generates significant press and interest.

2. Peak of Inflated Expectations. In the next phase, a frenzy of publicity typically generates over-enthusiasm and unrealistic expectations. There may be some successful applications of a technology, but there are typically more failures.

3. Trough of Disillusionment. Technologies enter the “trough of disillusionment” because they fail to meet expectations and quickly become unfashionable. Consequently, the press usually abandons the topic and the technology.

4. Slope of Enlightenment. Although the press may have stopped covering the technology, some businesses continue through the “slope of enlightenment” and experiment to understand the benefits and practical application of the technology.

5. Plateau of Productivity. A technology reaches the “plateau of productivity” as the benefits of it become widely demonstrated and accepted. The technology becomes increasingly stable and evolves in second and third generations. The final height of the plateau varies according to whether the technology is broadly applicable or benefits only a niche market.

Agile advocates promote “face-to-face conversations.” This, of course, is another name for a meeting. From a marketing perspective, it was smart of the Agile advocates not to use the word meeting in the Agile manifesto, but the reality remains.