What is Exhaustive Testing? And Why It Is Impossible?

What is Exhaustive Testing?

Exhaustive testing, which is also known as complete testing, occurs when all the testers in your team are exhausted and when all the planned tests have been executed. It is a quality assurance testing technique in which all scenarios or data is tested for testing. In a more understandable way, exhaustive testing means ensuring there are no undiscovered faults at the end of the test phase. Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. As testers, we often say “well I just never have enough time for testing”. Even if you had all the time in this world, you still wouldn’t have enough time to test all the possible input combinations and output combinations.

Why Exhaustive Testing is impractical and impossible?

It is not possible to perform complete testing or exhaustive testing. For most of the systems, it is near impossible because of the following reasons:

• The domain of possible inputs of a program is too large to be completely used in testing a system. There are both valid inputs and invalid inputs.
• The program may have a large number of states. There may be timing constraints on the inputs, that is, an input may be valid at a certain time and invalid at other times. An input value which is valid but is not properly timed is called an inopportune input.
• The input domain of a system can be very large to be completely used in testing a program.
• The design issues may be too complex to completely test. The design may have included implicit design decisions and assumptions. For example, a programmer may use a global variable or a static variable to control program execution.
• It may not be possible to create all possible execution environments of the system. This becomes more significant when the behavior of the software system depends on the real, outside world, such as weather, temperature, altitude, pressure, and so on.

Exhaustive Testing Examples

Example 1:

The IE Tools > Advanced Options window

53 binary conditions

1 condition with 3 options

1 condition with 4 options

2^53 = 9,007,199,254,740,992

       x                   12

       = 108,086,391,056,891,904 possible combinations of conditions

At one second per test execution:

108,086,391,056,891,904 / 360 = 300,239,975,158,033.067 hours (12,509,998,964,918.04 days or 34,273,969,766.9 years) to test all possible combinations.

Example 2:

Let's take an e-commerce site which has the following functionalities:

• Login
• Choose a product
  • Filter product with color
  • Filter a product with price.
• Add to Cart
• Buy the product (payment portal)

On the basis of risk identifying parameters, users can create a matrix to include in the test plan. Each parameter can be given scores so that we can have a correct way to identify high-risk areas.

• Impact on Business: 1-10
• Probability of Failure: 1-10
• Regression: 1-5
• Recovery: 1-5

Let’s create a matrix for the above example:

Functionality	Impact on Business	Probability of Failure	Regression	Recovery
Login	10	3	1	1=15
Choose a product with color filter	5	5	2	2=14
Choose a product with price filter	8	5	2	2=17
Add to Cart	10	8	3	4=25
Buy the product	10	7	2	2=21

So as per the score we have “Add to cart” functionality as the foremost contender for the "highest risk area" so now we can prioritize the testing. We can also determine for which functionalities QA team needs to do a near-exhaustive testing.
QA team can streamline the risk mitigation plan by looking at their scores

• Scores 1-5:- Unit testing and reviews.
• Scores 5-10:-Unit testing + black-box testing(regression and high business impact areas)
• Scores 10-15:-Typical test types with limited depth.
• Scores 15-25:-Typical test types with depth in certain test types.
• Scores 25-30:- High-risk areas. Complete coverage and in-depth testing for all test types.

As per the above-mentioned method areas with the score above 25 should be considered as an extremely high-risk area and a near-exhaustive testing needs to be done.So for the above example "Add to Cart" should implement in-depth testing for all test types and exhaustive testing should be done.

---

Source: https://testautomationresources.com/software-testing-basics/exhaustive-testing-fundamentals/