In the latest issues of Software Magazine, authors Kim Pries and Jon Quigley have presented a worthwhile approach to fit most software testing needs. The so-called four-phase approach concentrates on the following most used testing models: compliance, combinatorial, stochastic or exploratory, and extreme value testing. Below are the highlights of the article with most essential standpoints.
• To have compliance testing there should be requirements (product specifications) for which compliance is essential. That seems easy. But the challenging point is to define the product environment. This might not be as easy and in many cases also time-consuming and expensive. But on the other side of the story, underestimation of environmental factors, especially in embedded software development, may lead to big failures. If the product meets the customer specification but fails in the field, the customer organization will bear the burden of resulting penalty and suffer negative effects.
• Combinatorial testing. There is usually a combination of factors, environmental or external, present in the field which makes thorough adequate testing difficult. In some cases the most critical variable is time. How much time is necessary for thorough testing? For this, we need to evaluate substantial experimental or field information. Combinatorial testing consists in finding interactions of stimuli that can have impact on the product. The goal is to define these multiple stimuli that the product will most likely experience.
• Stochastic or exploratory testing occurs when an experienced test engineer uses his or her intuition to conduct unplanned experiments with the product. The value in this testing approach lies in the fact, that it adds a touch of reality by simulating the behavior of real users.
• Severe Environment testing. Extreme scenarios are those test cases that push the product to its limits. Relying on the commonly heard “this will never happen in the field” may lead to serious concerns, because events of the kind actually do usually occur in the field.
As a matter of fact, this is how a four-fold mechanism for providing substantial test coverage works for software development. A fifth optional method suggested by the authors is to attack own software as if you were evil “hackers” determined to break own code. The main point of the approach is to “attack” the product software in any possible way to see if and where it can be vulnerable. The only strong path to strong software is believed to be in looking not only for requirements failures but also for definable weaknesses.
Source: Softwaremag.com
• To have compliance testing there should be requirements (product specifications) for which compliance is essential. That seems easy. But the challenging point is to define the product environment. This might not be as easy and in many cases also time-consuming and expensive. But on the other side of the story, underestimation of environmental factors, especially in embedded software development, may lead to big failures. If the product meets the customer specification but fails in the field, the customer organization will bear the burden of resulting penalty and suffer negative effects.
• Combinatorial testing. There is usually a combination of factors, environmental or external, present in the field which makes thorough adequate testing difficult. In some cases the most critical variable is time. How much time is necessary for thorough testing? For this, we need to evaluate substantial experimental or field information. Combinatorial testing consists in finding interactions of stimuli that can have impact on the product. The goal is to define these multiple stimuli that the product will most likely experience.
• Stochastic or exploratory testing occurs when an experienced test engineer uses his or her intuition to conduct unplanned experiments with the product. The value in this testing approach lies in the fact, that it adds a touch of reality by simulating the behavior of real users.
• Severe Environment testing. Extreme scenarios are those test cases that push the product to its limits. Relying on the commonly heard “this will never happen in the field” may lead to serious concerns, because events of the kind actually do usually occur in the field.
As a matter of fact, this is how a four-fold mechanism for providing substantial test coverage works for software development. A fifth optional method suggested by the authors is to attack own software as if you were evil “hackers” determined to break own code. The main point of the approach is to “attack” the product software in any possible way to see if and where it can be vulnerable. The only strong path to strong software is believed to be in looking not only for requirements failures but also for definable weaknesses.
Source: Softwaremag.com
